Unable to download large file (Via byte[] or stream)

[Smurf-IV]

• Using V 2.1.6
• .NetCore 3.1
• Hosted in Chrome 87

I have a memory stream that is 280803689 bytes
I have direct access to the buffer it contains via var bytes = ms.GetBuffer();

I then call
DowloadFileResult fileResult = await DownloadFileService.DownloadFile(FileDetails.OriginalFileName, bytes, cts.Token, 1024*64, "application/octet-stream", OnProgress);
Then 6 minutes later I get this:

blazor.webassembly.js:1 crit: Microsoft.AspNetCore.Components.WebAssembly.Rendering.WebAssemblyRenderer[100]
      Unhandled exception rendering component: Out of memory
System.OutOfMemoryException: Out of memory
  at (wrapper managed-to-native) System.String.FastAllocateString(int)
  at System.String.CreateStringFromEncoding (System.Byte* bytes, System.Int32 byteLength, System.Text.Encoding encoding) <0x2fd3328 + 0x00030> in <filename unknown>:0 
  at System.Text.Encoding.GetString (System.Byte* bytes, System.Int32 byteCount) <0x2fd3170 + 0x0004e> in <filename unknown>:0 
  at System.Text.Json.JsonReaderHelper.TranscodeHelper (System.ReadOnlySpan`1[T] utf8Unescaped) <0x32b9e68 + 0x0004a> in <filename unknown>:0 
  at System.Text.Json.JsonSerializer.WriteCoreString (System.Object value, System.Type type, System.Text.Json.JsonSerializerOptions options) <0x356b5a0 + 0x00074> in <filename unknown>:0 
  at System.Text.Json.JsonSerializer.Serialize[TValue] (TValue value, System.Text.Json.JsonSerializerOptions options) <0x356b210 + 0x0000c> in <filename unknown>:0 
  at Microsoft.JSInterop.JSRuntime.InvokeAsync[TValue] (System.String identifier, System.Threading.CancellationToken cancellationToken, System.Object[] args) <0x3563820 + 0x001f4> in <filename unknown>:0 
  at Microsoft.JSInterop.JSRuntimeExtensions.InvokeVoidAsync (Microsoft.JSInterop.IJSRuntime jsRuntime, System.String identifier, System.Threading.CancellationToken cancellationToken, System.Object[] args) <0xc9b80a0 + 0x0004c> in <filename unknown>:0 
  at BlazorDownloadFile.BlazorDownloadFileService.DownloadFile (System.String fileName, System.Byte[] bytes, System.Threading.CancellationToken cancellationToken, System.Int32 bufferSize, System.String contentType, System.IProgress`1[T] progress) <0x4f21960 + 0x00246> in <filename unknown>:0 
  at System.Threading.Tasks.ValueTask`1[TResult].get_Result () <0x9ce4320 + 0x00034> in <filename unknown>:0 
  at ```

I call Garbage collect before calling the Download function to make sure there is plenty of available memory:

L: GC_MAJOR_SWEEP: major size: 2512K in use: 1523K
L: GC_MAJOR: (user request) time 63.28ms, stw 63.30ms los size: 450736K in use: 449168K

I have tried to use the `Memory stream` **directly** but it just drop 5MB of the file after about 3 minutes.
Also the `Progress Callback` is **only** called once !

[Smurf-IV]

I have tried via DownloadBinaryBuffers and DownloadBase64Buffers
But the calls for each 4MB (or 1MB) byte[] insertion take minutes each so is not performant enough

[arivera12]

This is a limitation of blazor and javascript itself I don't think it would be possible to fix browser max blob size. We would need to investigate futher on this.

[arivera12]

You may be hitting this

https://stackoverflow.com/questions/28307789/is-there-any-limitation-on-javascript-max-blob-size

[Smurf-IV]

I'm using this now..
https://www.meziantou.net/generating-and-downloading-a-file-in-a-blazor-webassembly-application.htm
and it works fine.

[arivera12]

Interesting article. He is using low level api and seems to gain more performance and less overhead. Thanks for the share. I will be considering changing the implementation in the near future.

[thalaeg]

I was having an issue with an even smaller size (400kb ish)

I turned my stream into a byte array and it worked.

@arivera12, you might want to impliment this

byte[] myByteArray;
using (var memoryStream = new MemoryStream())
{
    originalStream.CopyTo(memoryStream);
    myByteArray= memoryStream.ToArray();
}

for your

internal static async Task<byte[]> ToByteArrayAsync(this Stream stream)
{
    var streamLength = (int)stream.Length;
    var data = new byte[streamLength];
    stream.Position = 0;
    await stream.ReadAsync(data, 0, streamLength);
    return data;
}

I can submit a PR if you need/like

[arivera12]

@thalaeg which issue are you having? I don't get what are trying to tell me. This issue is about managing large files which I haven't worked on it yet. if you have another issue please create a new issue.

[thalaeg]

@arivera12 My issue was when I had a file that was too large (anything above 64ish KBs) the file would become corrupted.

I was downloading excel files (.xlsx extension) via a
await BlazorDownloadFileService.DownloadFile("usaFile.xlsx", usaStream, CancellationToken.None, CancellationToken.None, progress: null);

Where usaStream was a Stream.

When I changed usaStream to a byte[] via the method above, it worked and was no longer corrupted.

I can't share the file with you because of confidential info in it :(.

[arivera12]

that's weird are you using blazor server or blazor wasm?

if you are using blazor server you may be hitting this:

https://docs.microsoft.com/en-us/aspnet/core/signalr/security?view=aspnetcore-3.1#buffer-management

[thalaeg]

blazor wasm

here is the .csproj if helpful

<Project Sdk="Microsoft.NET.Sdk.BlazorWebAssembly">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <EmbeddedResource Include="data\CanadaDivisionCodes.json">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </EmbeddedResource>
    <EmbeddedResource Include="data\USADivisionCodes.json">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    </EmbeddedResource>
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="BlazorDownloadFile" Version="2.3.1" />
    <PackageReference Include="ClosedXML" Version="0.95.4" />
    <PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly" Version="6.0.0" />
    <PackageReference Include="Microsoft.AspNetCore.Components.WebAssembly.DevServer" Version="6.0.0" PrivateAssets="all" />
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
    <PackageReference Include="System.Net.Http.Json" Version="6.0.0">
      <TreatAsUsed>true</TreatAsUsed>
    </PackageReference>
    <PackageReference Include="MudBlazor" Version="6.0.2" />
  </ItemGroup>

    <ItemGroup>
      <Using Include="System" />
      <Using Include="System.Linq" />
      <Using Include="System.Threading.Tasks" />
      <Using Include="System.Collections.Generic" />
    </ItemGroup>

</Project>

[arivera12]

What I can think is that I may be having a bug in the stream overload method. I will move this into a new issue.