Skip to content

Latest commit

 

History

History
503 lines (440 loc) · 29.5 KB

FileReadAndWritePerformanceTests.md

File metadata and controls

503 lines (440 loc) · 29.5 KB

Writing & reading a file

There are a couple of classes that could be used to read & write data to & from a file:

  • File.Create returns a FileStream with a buffer size of 4096 and FileOptions.None.
  • File.CreateText & FileInfo.CreateText returns a StreamWriter with a buffer size of 1024 and FileOptions.SequentialScan.
  • File.ReadAllText uses a StreamReader with a buffer size of 1024 and FileOptions.SequentialScan.
  • StreamWriter & StreamReader are basically an additional buffer with default size 1024 added on top of an internal FileStream with buffer size 4096.
    • StreamWriter is able to write objects, different value types, strings and char arrays to a file.
    • Under the hood every input is converted to a string and then again converted to char array.
    • These char arrays fill the buffer and if it is full the buffer is converted to a byte array and written to the internal FileStream.
    • StreamReader reads all bytes from a file and converts them to a string.
  • Every way to access a file utilizes a FileStream which can only read & write bytes.

Setup

Old notebook

  • Intel Core i5-4200U - 2.3 GHz
  • DDR3 - 8GB RAM - 1600 MHz
  • Toshiba MQ01ABF050 - Read 100 MB/s Write 96 MB/s

First test: Comparison of the different classes

The test will write 10 MB to a file:

  • Inputs
    • a single string
    • a char array
    • a byte array
  • Block sizes (and buffer size for FileStream)
    • 4096
    • 8192
    • 16384
    • 32768
    • 65536
  • The elapsed time is the average of 12 tests as the fractional portion of a second.
  • Abbreviations:
    • File.CT = File.CreateText
    • File.C = File.Create
    • SW = StreamWriter
    • FS = FileStream
  • (Complete results)

StreamWriter:

Position Class Buffer Block Write calls FileOption Time reduction in % Time
Base line File.CT + string 1024 1293107 1 FileOptions.SequentialScan 00:1293548
1. SW + char array 4096 1221 8192 FileOptions.None -14.01 00:1111940
2. File.CT + char array 1024 1221 8192 FileOptions.None -13.82 00:1114435
3. FileInfo + char array 32768 2442 4096 FileOptions.SequentialScan -13.64 00:1116693

Note

StreamWriter was only tested with the default buffer size of 4096 after comparing it with the FileStream default.

FileStream:

Position Class Buffer Block Write calls FileOption Time reduction in % Time
1. FS 16384 153 65536 FileOptions.None -96.04 00:0051233
2. File.C 32768 153 65536 FileOptions.None -95.97 00:0052066
3. File.C 65536 153 65536 FileOptions.None -95.93 00:0052638

Note

FileStream is clearly faster but finding the correct buffer & block size combinations seems rather tricky.

Simple explanation of the internal logic of a FileStream:

  • Has an internal buffer which holds the bytes. If the buffer is full it is written to the file system cache.
    • This can be forced by calling FileStream.Flush. This is also called when disposing the FileStream.
  • The file system writes the data lazily to disk depending on the hard disk write speed.
  • Reading the bytes is similiar. Depending on the FileOption more or less is cached by the file systen which in turn allows for faster sequential paging through the data or faster access at random positions.
  • Read & write have a couple of options which can be added when calling the methods. For this case the following are interesting:
    • FileOptions.None will pass no additional access flags to the file system. So it will try to guess the optimal cache size depending on the access pattern.
    • FileOptions.RandomAccess will cache less because it expects the file to be accessed at random positions by multiple applications.
    • FileOptions.SequentialScan will cache more because it expects the file to be accessed sequentially by a single application.
    • FileOptions.WriteThrough will write to file system cache but is flushed to disk without delay.
    • FILE_FLAG_NO_BUFFERING will ignore the file system cache but has has special memory alignment requierements.
      • It isn't included in FileOptions but can still be passed down with 0x20000000.
      • File, buffer & block size need to be an integer multiple of 512 bytes.
    • If FILE_FLAG_NO_BUFFERING is used in combination with FileOptions.WriteThrough the file system cache is ignored and the data is immediately flushed to disk.

Second test: FileStream, buffer and block size

The test will use the FileStream to write & read 30 MB to & from a file:

  • Buffer and block sizes
    • 1024
    • 2048
    • 4096
    • 8192
    • 16384
    • 32768
    • 65536
    • 131072
    • 262144
  • FileOption
    1. FileOptions.None
    2. FileOptions.SequentialScan
    3. FileOptions.WriteThrough
    4. FileOptions.SequentialScan + FileOptions.WriteThrough
    5. FILE_FLAG_NO_BUFFERING
    6. FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING
    7. FileOptions.SequentialScan + FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING
  • The elapsed time is the median of 12 tests as the fractional portion of a second.
    • The elapsed times of each test are sorted, first and last quarter is ignored to avoid using the extrem values and then the average is calcualted from the remaining half.
  • Executed code
    • .NET Framework 4.7.2 Console with byte[]
    • .NET 5 (Core) Console with Spans

Write

  • .NET Framework 4.7.2 + byte array

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 FileOptions.None 00:1299553
    1. 1024 262144 115 FileOptions.SequentialScan -90.62 00:0121946
    2. 131072 262144 115 FileOptions.SequentialScan -90.55 00:0122786
    3. 32768 262144 115 FileOptions.SequentialScan -90.49 00:0123639
    4. 8192 262144 115 FileOptions.None -90.43 00:0124345
    5. 16384 262144 115 FileOptions.SequentialScan -90.31 00:0125923
    6. 65536 262144 115 FileOptions.None -90.29 00:0126154
    7. 2048 262144 115 FileOptions.SequentialScan -90.24 00:0126871
    8. 131072 262144 115 FileOptions.None -90.11 00:0128578
    9. 65536 262144 115 FileOptions.SequentialScan -90.07 00:0129060
    10. 4096 262144 115 FileOptions.None -89.86 00:0131826
    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 File_FLAG_NO_BUFFERING 14:0765745
    1. 32768 262144 115 File_FLAG_NO_BUFFERING -95.46 00:6384932
    2. 8192 262144 115 File_FLAG_NO_BUFFERING -95.46 00:6393680
    3. 262144 262144 115 File_FLAG_NO_BUFFERING -95.35 00:6548806
    4. 1024 262144 115 File_FLAG_NO_BUFFERING -95.16 00:6818550
    5. 65536 262144 115 File_FLAG_NO_BUFFERING -95.04 00:6987442
    6. 131072 262144 115 FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -95.01 00:7022802
    7. 4096 262144 115 File_FLAG_NO_BUFFERING -94.99 00:7058127
    8. 2048 262144 115 File_FLAG_NO_BUFFERING -94.95 00:7113264
    9. 16384 262144 115 File_FLAG_NO_BUFFERING -94.83 00:7273666
    10. 262144 8192 3663 File_FLAG_NO_BUFFERING -94.79 00:7328590

  • .NET 5 (Core) + byte spans

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 FileOptions.None 00:1514422
    1. 4096 262144 115 FileOptions.None -91.69 00:0125792
    2. 32768 262144 115 FileOptions.SequentialScan -91.69 00:0125872
    3. 262144 262144 115 FileOptions.SequentialScan -91.61 00:0127124
    4. 8192 262144 115 FileOptions.SequentialScan -91.51 00:0128543
    5. 4096 262144 115 FileOptions.SequentialScan -91.46 00:0129328
    6. 1024 262144 115 FileOptions.SequentialScan -91.39 00:0130422
    7. 32768 262144 115 FileOptions.None -91.36 00:0130781
    8. 16384 262144 115 FileOptions.SequentialScan -91.12 00:0134531
    9. 65536 262144 115 FileOptions.SequentialScan -91.10 00:0134734
    10. 16384 262144 115 FileOptions.None -91.09 00:0134885
    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 File_FLAG_NO_BUFFERING 08:5941065
    1. 262144 8192 3663 File_FLAG_NO_BUFFERING -93.98 00:5170241
    2. 8192 262144 115 File_FLAG_NO_BUFFERING -93.97 00:5183650
    3. 131072 262144 115 File_FLAG_NO_BUFFERING -93.82 00:5312722
    4. 2048 262144 115 File_FLAG_NO_BUFFERING -93.82 00:5315240
    5. 131072 262144 115 FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -93.76 00:5358676
    6. 262144 32768 916 File_FLAG_NO_BUFFERING -93.71 00:5401459
    7. 262144 2048 14649 File_FLAG_NO_BUFFERING -93.69 00:5421775
    8. 16384 262144 115 File_FLAG_NO_BUFFERING -93.67 00:5440487
    9. 262144 262144 115 File_FLAG_NO_BUFFERING -93.67 00:5443361
    10. 262144 65536 458 File_FLAG_NO_BUFFERING -93.62 00:5487314

  • Conclusion

    • For FileOption 1. - 4. both .NET versions had the best results with block size 262144 & FileOptions.SequentialScan but the buffer sizes is random. There seems to be no major difference in speed between using byte arrays & spans of bytes when writing the data to the stream but in total the .NET Framework version ran slightly faster looking at the actual execution times.
      • The benefit of spans is less memory allocation, faster access & modification of the underlying memory. A difference between byte array & spans might show later in the finished code.
      • Need to test larger buffer & block sizes.
      • For the currently used versions FileOption 3. & 4. can be ignored for future tests. Might have better results in newer .NET versions.
    • For FileOption 5. - 7. both .NET versions had the best results with block size 262144 & FILE_FLAG_NO_BUFFERING but the buffer sizes is random.
      • .NET 5 performed better than .NET Framework but in comparison with 1. & 2. FileOption they were really slow.
      • For the currently used versions FileOption 5. - 7. can be ignored for future tests. Might have better results in newer .NET versions.

Read

  • .NET Framework 4.7.2 + byte array

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 FileOptions.None 00:0693528
    1. 8192 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough -87.30 00:0088072
    2. 1024 131072 229 FileOptions.SequentialScan + FileOptions.WriteThrough -87.22 00:0088628
    3. 131072 131072 229 FileOptions.SequentialScan + FileOptions.WriteThrough -87.18 00:0088898
    4. 16384 262144 115 FileOptions.WriteThrough -87.16 00:0089060
    5. 131072 131072 229 FileOptions.WriteThrough -87.14 00:0089171
    6. 1024 262144 115 FileOptions.WriteThrough -87.14 00:0089188
    7. 65536 262144 115 FileOptions.WriteThrough -87.14 00:0089192
    8. 32768 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough -87.12 00:0089313
    9. 2048 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough -87.12 00:0089314
    10. 32768 262144 115 FileOptions.WriteThrough -87.12 00:0089352
    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 File_FLAG_NO_BUFFERING 07:4466310
    1. 8192 262144 115 _FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -95.22 00:3555872
    2. 65536 262144 115 File_FLAG_NO_BUFFERING -95.08 00:3666949
    3. 262144 65536 458 FileOptions.SequentialScan + FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -95.08 00:3666935
    4. 262144 131072 229 FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -95.08 00:3667076
    5. 65536 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -95.08 00:3667002
    6. 262144 32768 916 FileOptions.SequentialScan + FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -94.93 00:3778057
    7. 1024 262144 115 File_FLAG_NO_BUFFERING -94.93 00:3778061
    8. 16384 262144 115 FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -94.93 00:3778302
    9. 131072 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -94.78 00:3889167
    10. 2048 262144 115 FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -94.78 00:3889174

  • .NET 5 (Core) + byte spans

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 FileOptions.None 00:0704307
    1. 2048 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough -88.12 00:0083699
    2. 1024 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough -87.61 00:0087258
    3. 1024 262144 115 FileOptions.WriteThrough -87.52 00:0087911
    4. 2048 262144 115 FileOptions.WriteThrough -87.48 00:0088184
    5. 65536 131072 229 FileOptions.WriteThrough -87.48 00:0088207
    6. 4096 65536 458 FileOptions.SequentialScan + FileOptions.WriteThrough -87.47 00:0088221
    7. 32768 65536 458 FileOptions.SequentialScan + FileOptions.WriteThrough -87.45 00:0088392
    8. 32768 131072 229 FileOptions.WriteThrough -87.44 00:0088446
    9. 1024 65536 458 FileOptions.SequentialScan + FileOptions.WriteThrough -87.43 00:0088549
    10. 2048 131072 229 FileOptions.SequentialScan + FileOptions.WriteThrough -87.41 00:0088656
    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 File_FLAG_NO_BUFFERING 07:4339171
    1. 8192 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -95.17 00:3593126
    2. 16384 262144 115 FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -95.07 00:3667849
    3. 131072 262144 115 File_FLAG_NO_BUFFERING -94.92 00:3777291
    4. 65536 262144 115 FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -94.92 00:3778078
    5. 262144 262144 115 FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -94.77 00:3889190
    6. 8192 262144 115 File_FLAG_NO_BUFFERING -94.77 00:3889193
    7. 16384 262144 115 File_FLAG_NO_BUFFERING -94.77 00:3889197
    8. 262144 131072 229 FileOptions.SequentialScan + FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING -94.77 00:3889210
    9. 1024 262144 115 File_FLAG_NO_BUFFERING -94.77 00:3889253
    10. 262144 262144 115 File_FLAG_NO_BUFFERING -94.77 00:3889264

  • Conclusion

    • For FileOption 1. - 4. both .NET versions had the best results with block size 262144 or 131072 & FileOptions.SequentialScan + FileOptions.WriteThrough or FileOptions.WriteThrough but the buffer sizes is random. There seems to be no major difference in speed between using byte arrays & spans of bytes when reading the data from the stream but in total the .NET 5 version ran slightly faster looking at the actual execution times.
      • The benefit of spans is less memory allocation, faster access & modification of the underlying memory. A difference between byte array & spans might show later in the finished code.
      • Need to test larger block & Block sizes.
      • For the currently used versions FileOption 1. & 2. can be ignored for future tests. Might have better results in newer .NET versions.
    • For FileOption 5. - 7. both .NET versions had the best results with block size 262144 & FileOptions.SequentialScan + FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING for .NET Framework or FileOptions.WriteThrough + FILE_FLAG_NO_BUFFERING for .NET 5 but the buffer sizes is random.
      • Both versions performed similiar but in comparison with 3. & 4. FileOption they were really slow.
      • For the currently used versions FileOption 5. - 7. can be ignored for future tests. Might have better results in newer .NET versions.

Third & fourth test: FileStream.Write, bigger buffer and block size

The test will use the FileStream to write & read 30 MB to & from a file:

  • Buffer & block sizes
    • 1024
    • 2048
    • 4096
    • 8192
    • 16384
    • 32768
    • 65536
    • 131072
    • 262144
    • 524288
    • 1048576
    • 2097152
    • 4194304
    • 8388608
    • 16777216
    • 33554432
  • FileOption
    • write
      1. FileOptions.None
      2. FileOptions.SequentialScan
    • Read
      1. FileOptions.None
      2. FileOptions.SequentialScan
      3. FileOptions.WriteThrough
      4. FileOptions.SequentialScan + FileOptions.WriteThrough
  • The elapsed time is the median of 12 tests as the fractional portion of a second.
    • The elapsed times of each test are sorted, first and last quarter is ignored to avoid using the extrem values and then the average is calcualted from the remaining half.
  • Executed code
    • .NET Framework 4.7.2 Console with byte[]
    • .NET 5 (Core) Console with Spans

Note

The third test was executed with a bigger block size than the second but still didn't provide a clear result. So the fourth test was executed with an even bigger block & buffer sizes. The results from the third test can be found here.

Write

  • .NET Framework 4.7.2 + byte array + FileOption 1. + 2. (Complete results)

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 FileOptions.None 00:1278846
    1. 4194304 16777216 2 FileOptions.None -90.76 00:0118136
    2. 524288 16777216 2 FileOptions.None -90.70 00:0118953
    3. 32768 16777216 2 FileOptions.None -90.60 00:0120170
    4. 16384 4194304 8 FileOptions.SequentialScan -90.60 00:0120193
    5. 1024 2097152 15 FileOptions.SequentialScan -90.58 00:0120462
    6. 524288 16777216 2 FileOptions.SequentialScan -90.55 00:0120898
    7. 2048 16777216 2 FileOptions.SequentialScan -90.54 00:0120928
    8. 262144 2097152 15 FileOptions.SequentialScan -90.52 00:0121239
    9. 2048 4194304 8 FileOptions.None -90.51 00:0121376
    10. 262144 2097152 15 FileOptions.None -90.49 00:0121654

  • .NET 5 (Core) + byte spans + FileOption 1. + 2. (Complete results)

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 FileOptions.None 00:1283446
    1. 32768 2097152 15 FileOptions.SequentialScan -90.57 00:0120974
    2. 262144 16777216 2 FileOptions.SequentialScan -90.52 00:0121641
    3. 2097152 16777216 2 FileOptions.None -90.50 00:0121873
    4. 8192 2097152 15 FileOptions.None -90.46 00:0122399
    5. 131072 4194304 8 FileOptions.None -90.40 00:0123180
    6. 262144 16777216 2 FileOptions.None -90.37 00:0123553
    7. 1024 2097152 15 FileOptions.SequentialScan -90.37 00:0123637
    8. 1048576 4194304 8 FileOptions.None -90.37 00:0123639
    9. 2048 524288 58 FileOptions.SequentialScan -90.35 00:0123818
    10. 1048576 1048576 29 FileOptions.SequentialScan -90.35 00:0123835

  • Conclusion

    • Buffer size is a bit random but 262144 occurred 4 times. The values mostly have a block size which is 4 - 32 times the buffer size.
    • Previous block size was 262144 but now the most occuring values are 8 times 16777216, 6 times 2097152 and 4 times 4194304. These values are 64, 8 and 16 times the previous value.
    • Previous FileOption was FileOptions.SequentialScan but now split between FileOptions.None and FileOptions.SequentialScan.
    • After this test the best combination would be:
      • Buffer size 262144
      • Block size 16777216
      • FileOptions.SequentialScan

Read

  • .NET Framework 4.7.2 + byte array + FileOption 1. - 4. (Complete results)

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 FileOptions.None 00:0677960
    1. 262144 262144 115 FileOptions.SequentialScan -86.53 00:0091300
    2. 262144 262144 115 FileOptions.SequentialScan + FileOptions.WriteThrough -86.46 00:0091799
    3. 2048 262144 115 FileOptions.WriteThrough -86.15 00:0093867
    4. 262144 262144 115 FileOptions.WriteThrough -86.14 00:0093975
    5. 2048 524288 58 FileOptions.WriteThrough -86.10 00:0094265
    6. 4096 65536 458 FileOptions.None -86.03 00:0094686
    7. 1024 262144 115 FileOptions.None -86.02 00:0094799
    8. 2048 262144 115 FileOptions.SequentialScan -85.99 00:0094968
    9. 131072 131072 229 FileOptions.SequentialScan + FileOptions.WriteThrough -85.95 00:0095231
    10. 2048 524288 58 FileOptions.SequentialScan + FileOptions.WriteThrough -85.94 00:0095298

  • .NET 5 (Core) + byte spans + FileOption 1. - 4. (Complete results)

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 1024 1024 29297 FileOptions.None 00:0633799
    1. 8192 131072 229 FileOptions.SequentialScan -86.56 00:0085192
    2. 2048 131072 229 FileOptions.SequentialScan + FileOptions.WriteThrough -86.34 00:0086548
    3. 8192 4194304 58 FileOptions.SequentialScan -86.20 00:0087436
    4. 4096 131072 229 FileOptions.SequentialScan + FileOptions.WriteThrough -86.13 00:0087930
    5. 4096 4194304 58 FileOptions.SequentialScan + FileOptions.WriteThrough -86.12 00:0087960
    6. 8192 131072 229 FileOptions.SequentialScan + FileOptions.WriteThrough -86.12 00:0087976
    7. 4096 2097152 58 FileOptions.SequentialScan -86.08 00:0088203
    8. 16384 131072 229 FileOptions.WriteThrough -85.95 00:0089049
    9. 8192 131072 229 FileOptions.WriteThrough -85.57 00:0091463
    10. 16384 131072 229 FileOptions.None -85.53 00:0091731

  • Conclusion

    • Buffer size is a bit random but 2048 on .NET Framework and 8192 on .NET Core occured the most.
    • Previous block size was 262144 but now the most occuring values are 6 times 262144 on .NET Framework and 7 times 131072 on .NET Core.
    • FileOption remained mostly FileOptions.SequentialScan + FileOptions.WriteThrough.
    • After this test the best combination would be:
      • Buffer size 2048 or 8192
      • Block size 262144 or 131072
      • FileOptions.SequentialScan + FileOptions.WriteThrough

Fifth test: FileStream.Write, buffer and block size between 5-100% L1 cache size

The test will use the FileStream to write & read 30 MB to & from a file:

  • Buffer & block sizes (5% steps of L1 cache size)
    • 6400
    • 12800
    • 19200
    • 25600
    • 32000
    • 38400
    • 44800
    • 51200
    • 57600
    • 64000
    • 70400
    • 76800
    • 83200
    • 89600
    • 96000
    • 102400
    • 108800
    • 115200
    • 121600
    • 128000
  • FileOption
    • write
      1. FileOptions.None
      2. FileOptions.SequentialScan
    • Read
      1. FileOptions.None
      2. FileOptions.SequentialScan
      3. FileOptions.WriteThrough
      4. FileOptions.SequentialScan + FileOptions.WriteThrough
  • The elapsed time is the median of 12 tests as the fractional portion of a second.
    • The elapsed times of each test are sorted, first and last quarter is ignored to avoid using the extrem values and then the average is calcualted from the remaining half.
  • Executed code
    • .NET Framework 4.7.2 Console with byte[]
    • .NET 5 (Core) Console with Spans

Write

  • .NET Framework 4.7.2 + byte array + FileOption 1. + 2. (Complete results)

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 6400 6400 4688 FileOptions.None 00:0542895
    1. 19200 102400 293 FileOptions.None -74.78 00:0136921
    2. 44800 102400 293 FileOptions.None -74.53 00:0138276
    3. 70400 108800 276 FileOptions.None -74.36 00:0139213
    4. 128000 128000 235 FileOptions.None -74.34 00:0139311
    5. 115200 121600 247 FileOptions.SequentialScan -74.23 00:0139888
    6. 19200 102400 293 FileOptions.SequentialScan -74.20 00:0140078
    7. 64000 102400 293 FileOptions.SequentialScan -74.15 00:0140361
    8. 44800 102400 293 FileOptions.SequentialScan -74.06 00:0140812
    9. 57600 121600 247 FileOptions.None 74.00 00:0141161
    10. 25600 115200 261 FileOptions.SequentialScan -73.90 00:0141719

  • .NET 5 (Core) + byte spans + FileOption 1. + 2. (Complete results)

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 6400 6400 4688 FileOptions.None 00:0551297
    1. 19200 102400 293 FileOptions.SequentialScan -74.60 00:0140005
    2. 12800 102400 293 FileOptions.None -74.58 00:0140133
    3. 44800 102400 293 FileOptions.None -74.34 00:0141486
    4. 51200 121600 247 FileOptions.SequentialScan -74.07 00:0142970
    5. 70400 102400 293 FileOptions.None -74.01 00:0143256
    6. 6400 115200 261 FileOptions.None -73.92 00:0143784
    7. 51200 128000 235 FileOptions.None -73.91 00:0143812
    8. 64000 102400 293 FileOptions.SequentialScan -73.84 00:0144239
    9. 44800 102400 293 FileOptions.SequentialScan -73.79 00:0144521
    10. 96000 102400 293 FileOptions.None -73.69 00:0145029

Read

  • .NET Framework 4.7.2 + byte array + FileOption 1. - 4. (Complete results)

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 6400 6400 4688 FileOptions.None 00:0184818
    1. 57600 83200 361 FileOptions.SequentialScan + FileOptions.WriteThrough -53.39 00:0086139
    2. 38400 89600 335 FileOptions.SequentialScan + FileOptions.WriteThrough -53.10 00:0086677
    3. 83200 128000 235 FileOptions.SequentialScan + FileOptions.WriteThrough -52.83 00:0087186
    4. 38400 89600 335 FileOptions.WriteThrough -52.81 00:0087207
    5. 83200 128000 235 FileOptions.SequentialScan -52.77 00:0087290
    6. 64000 83200 361 FileOptions.SequentialScan + FileOptions.WriteThrough -52.61 00:0087578
    7. 83200 128000 235 FileOptions.WriteThrough -52.61 00:0087594
    8. 76800 128000 235 FileOptions.SequentialScan + FileOptions.WriteThrough -52.59 00:0087623
    9. 6400 57600 521 FileOptions.SequentialScan + FileOptions.WriteThrough -52.47 00:0087839
    10. 51200 83200 361 FileOptions.SequentialScan + FileOptions.WriteThrough -52.45 00:0087884

  • .NET 5 (Core) + byte spans + FileOption 1. - 4. (Complete results)

    Position Buffer Block Write calls FileOption Time reduction in % Time
    Base line 6400 6400 4688 FileOptions.None 00:0182918
    1. 38400 83200 361 FileOptions.SequentialScan + FileOptions.WriteThrough -53.75 00:0084599
    2. 38400 83200 361 FileOptions.WriteThrough -53.37 00:0085303
    3. 44800 70400 427 FileOptions.WriteThrough -53.34 00:0085350
    4. 38400 57600 521 FileOptions.WriteThrough -52.74 00:0086455
    5. 51200 64000 469 FileOptions.SequentialScan -52.61 00:0086676
    6. 38400 57600 521 FileOptions.SequentialScan + FileOptions.WriteThrough -52.61 00:0086688
    7. 51200 83200 361 FileOptions.SequentialScan + FileOptions.WriteThrough -52.15 00:0087524
    8. 12800 64000 469 FileOptions.SequentialScan + FileOptions.WriteThrough -51.86 00:0088060
    9. 44800 70400 427 FileOptions.SequentialScan + FileOptions.WriteThrough -51.86 00:0088065
    10. 76800 115200 261 FileOptions.WriteThrough -51.74 00:0088285

Note

The process to find the optimal combination is extremly time consuming and it is not realistic to run this on every device the read / write code should run on. A quick test with 5-100% of the L1 cache size showed that a result close to the optimal can be archieved if a buffer size of 25-50% and a block size of 60-80% of the L1 cache size is choosen. The FileOptions are still FileOptions.SequentialScan for write and FileOptions.SequentialScan + FileOptions.WriteThrough for read.

Tip

Additional informations on the topic: