Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to download a file larger than 500 MB #39

Open
DenisPolagaev opened this issue May 24, 2022 · 21 comments
Open

Unable to download a file larger than 500 MB #39

DenisPolagaev opened this issue May 24, 2022 · 21 comments
Assignees
Labels
good first issue Good for newcomers

Comments

@DenisPolagaev
Copy link

Unable to download a file larger than 500 MB.

Used your library, in particular the method: DownloadFile(string filename, Stream stream, CancellationToken cancellationTokenBytesRead, CancellationToken cancellationTokenJavaScriptInterop, string ContentType = "application/octet-stream").

But when you try to download a file larger than 200 MB, an exception occurs, which says that there was not enough memory. Stream was too long

Question on Stackoverflow: https://stackoverflow.com/questions/72356882/generate-a-link-to-the-download-file

@arivera12
Copy link
Owner

arivera12 commented May 24, 2022

This method you mentioned above takes the entire stream as pass it to javascript entirely, and since your file is too big this error is expected.

You need to use the overload method with the bufferSize to pass it by chunks to javascript.

More bigger is the bufferSize then the download speed is better.

@arivera12 arivera12 self-assigned this May 24, 2022
@arivera12 arivera12 added the good first issue Good for newcomers label May 24, 2022
@DenisPolagaev
Copy link
Author

Thanks for the answer, I'll try in 5 hours, I'll unsubscribe.

@DenisPolagaev
Copy link
Author

DenisPolagaev commented May 26, 2022

I use this code, but it downloads a piece of buffer 1 time, then nothing happens.

var fileInfo = new System.IO.FileInfo(filePath);
if (fileInfo.Exists)
{
     var memory = new MemoryStream();
     using FileStream fsSource = new(filePath, FileMode.Open, FileAccess.Read);

     //fsSource.CopyTo(memory);
     await downloadService.DownloadFile(file.FileName, fsSource , 32768, contentType: "application/octet-stream");
}

@DenisPolagaev
Copy link
Author

DenisPolagaev commented May 26, 2022

Or here's another:

using FileStream fsSource = new(filePath,FileMode.Open,  FileAccess.Read);
                            await downloadService.DownloadFile(file.FileName, fsSource, 32768, contentType: "application/octet-stream", async (p) =>
  {
                                Console.WriteLine(p);
                                await Task.Delay(0);
 });

FileSize:

image

Stream size:
272445978

Console.WriteLine Output:

0,00012027338498643573
0,00024054676997287147
0,0003608201549593072
0,00048109353994574294
0,0006013669249321787
0,0007216403099186144
0,0008419136949050502
0,0009621870798914859
0,0010824604648779217
0,0012027338498643574
0,001323007234850793
0,0014432806198372288
0,0015635540048236645
0,0016838273898101004
0,001804100774796536
0,0019243741597829718
0,0020446475447694077
0,0021649209297558434
0,002285194314742279
0,0024054676997287147
0,0025257410847151504
0,002646014469701586
0,002766287854688022
0,0028865612396744575
0,0030068346246608932
0,003127108009647329
0,0032473813946337646
0,0033676547796202007
0,0034879281646066364
0,003608201549593072
0,003728474934579508
0,0038487483195659435
0,003969021704552379
0,004089295089538815
0,004209568474525251
0,004329841859511687
0,004450115244498122
0,004570388629484558
0,004690662014470994
0,0048109353994574295
0,004931208784443865
0,005051482169430301
0,005171755554416737
0,005292028939403172
0,005412302324389608
0,005532575709376044
0,005652849094362479
0,005773122479348915
0,005893395864335351
0,0060136692493217864
0,006133942634308222
0,006254216019294658
0,0063744894042810935
0,006494762789267529
0,006615036174253965
0,0067353095592404015
0,006855582944226837
0,006975856329213273
0,0070961297141997086
0,007216403099186144
0,00733667648417258
0,007456949869159016
0,007577223254145451
0,007697496639131887
0,007817770024118322
0,007938043409104758
0,008058316794091195
0,00817859017907763
0,008298863564064066
0,008419136949050502
0,008539410334036938
0,008659683719023373
0,00877995710400981
0,008900230488996245
0,00902050387398268
0,009140777258969116
0,009261050643955552
0,009381324028941988
0,009501597413928423
0,009621870798914859
0,009742144183901295
0,00986241756888773
0,009982690953874166
0,010102964338860602
0,010223237723847037
0,010343511108833473
0,010463784493819909
0,010584057878806345
0,01070433126379278
0,010824604648779216
0,010944878033765652
0,011065151418752087
0,011185424803738523
0,011305698188724959
0,011425971573711394
0,01154624495869783
0,011666518343684266
0,011786791728670701
0,011907065113657137
0,012027338498643573
0,012147611883630009
0,012267885268616444
0,01238815865360288
0,012508432038589316
0,012628705423575751
0,012748978808562187
0,012869252193548623
0,012989525578535058
0,013109798963521494
0,01323007234850793
0,013350345733494366
0,013470619118480803
0,013590892503467239
0,013711165888453674
0,01383143927344011
0,013951712658426546
0,014071986043412981
0,014192259428399417
0,014312532813385853
0,014432806198372288
0,014553079583358724
0,01467335296834516
0,014793626353331596
0,014913899738318031
0,015034173123304467
0,015154446508290903
0,015274719893277338
0,015394993278263774
0,01551526666325021

And download

image

@DenisPolagaev
Copy link
Author

DenisPolagaev commented May 26, 2022

I user you code:

await jsRuntime.InvokeVoidAsync("_blazorDownloadFileClearBuffer");
using FileStream fsSource = new(filePath,FileMode.Open,  FileAccess.Read);
 var totalOfBytes = (int)fsSource.Length;
var totalOfBytesReaded = 0;
var pendingBytesToRead = totalOfBytes;
 do
 {
       var currentBufferSize = 32768 > totalOfBytes ? totalOfBytes : 32768 > pendingBytesToRead ? pendingBytesToRead : 32768;
        var buffer = new byte[currentBufferSize];
        totalOfBytesReaded += await fsSource.ReadAsync(buffer.AsMemory(0, currentBufferSize));
        pendingBytesToRead -= totalOfBytesReaded;

          await jsRuntime.InvokeVoidAsync("_blazorDownloadFileBuffersPush", buffer);
} while (pendingBytesToRead > 0);

var fileResult = await jsRuntime.InvokeAsync<DownloadFileResult>("_blazorDowloadFileByteArrayPartitioned", fileInfo.Name, "application/octet-stream");
Console.WriteLine(fileResult.ErrorName);
Console.WriteLine(fileResult.Succeeded); // True

@arivera12
Copy link
Owner

Where is this issue ocurring?

Blazor server?
Blazor wasm?

Which browser and version?

@arivera12
Copy link
Owner

arivera12 commented May 26, 2022

Also need to know which arq is your cpu and OS and how much memory you computer has since there are limitations when downloading big files through javascript per browser provider.

@arivera12
Copy link
Owner

Please read this and see if this case applies to your case.
https://stackoverflow.com/questions/28307789/is-there-any-limitation-on-javascript-max-blob-size

@DenisPolagaev
Copy link
Author

image

@DenisPolagaev
Copy link
Author

DenisPolagaev commented May 26, 2022

I did it like this. It works.

await jsRuntime.InvokeVoidAsync("_blazorDownloadFileClearBuffer");
using FileStream fsSource = new(filePath, FileMode.Open, FileAccess.Read);

var buffer = new byte[2097152];
int bytesRead = await fsSource.ReadAsync(buffer);
await jsRuntime.InvokeVoidAsync("_blazorDownloadFileBuffersPush", buffer);
while (bytesRead > 0)
{
         bytesRead = await fsSource.ReadAsync(buffer.AsMemory(0, 2097152));
         await jsRuntime.InvokeVoidAsync("_blazorDownloadFileBuffersPush", buffer);
}
var fileResult = await jsRuntime.InvokeAsync<DownloadFileResult>("_blazorDowloadFileByteArrayPartitioned", fileInfo.Name, "application/octet-stream");
if (fileResult.Succeeded)
{
             _notificationService.Notify(new NotificationMessage { Severity = NotificationSeverity.Success, Summary = "Инфо", Detail = $"{file.FileName} отправлен на скачивание.", Duration = 4000 });
}
fsSource.Close();
await jsRuntime.InvokeVoidAsync("_blazorDownloadFileClearBuffer");

But there is a nuance, when the user clicked on the download button, he first downloads the buffer, and only then the save appears.

image

@DenisPolagaev
Copy link
Author

Where is this issue ocurring?

Blazor server? Blazor wasm?

Which browser and version?

Latest Yandex, Opera.

Blazor Server

@DenisPolagaev
Copy link
Author

DenisPolagaev commented May 26, 2022

And to download the file, it shakes in chunks, while generating requests.

image

per 552 mb file, 804 requests

@DenisPolagaev
Copy link
Author

In js, can it be first to give the user a link, and then write bytes there?

@arivera12
Copy link
Owner

And to download the file, it shakes in chunks, while generating requests.

image

per 552 mb file, 804 requests

This is the way it works, and there is nothing else we can do about it.

You may need to increase the buffer size as you seems to do it already to lower the requests number.

Remember that blazor is transferring the bytes in chunks from blazor server to javascript via js interop.

@arivera12
Copy link
Owner

In js, can it be first to give the user a link, and then write bytes there?

what you mean by this?

@arivera12
Copy link
Owner

arivera12 commented May 26, 2022

The trick over here is increase the buffer size.

Blazor server signal r should be halting from so many request on so short time span.

Increase the buffersize to 2-5 megabytes and try again for that download.

@DenisPolagaev
Copy link
Author

In ASP net core, they used to do this:

https://stackoverflow.com/questions/42460198/return-file-in-asp-net-core-web-api

In the Blazor, you can give a link to the file https://docs.microsoft.com/en-us/aspnet/core/blazor/file-downloads ?view=aspnetcore-6.0

@DenisPolagaev
Copy link
Author

The trick over here is increase the buffer size.

Blazor server signal r should be halting from so many request on so short time span.

Increase the buffersize to 2-5 megabytes and try again for that download.

I'll try it tomorrow, but I think I can find an alternative. And do , for example , how to give files to Google Dock . Generate a link, the user receives a link to the stream, the browser understands and downloads.

@arivera12
Copy link
Owner

The best option for big file is use the content-disposition attachment response as I explained on the readme.md of this repository.

image

@arivera12
Copy link
Owner

And yes, even if I recommend use content-disposition from files incoming from the server, you can still download files from the server as you are trying to do over my library but huge files incoming from the server as is your case will halt blazor server or javascript max blob size on the browser side if it's extremely big or you dont tune up the buffersize.

@arivera12
Copy link
Owner

In ASP net core, they used to do this:

https://stackoverflow.com/questions/42460198/return-file-in-asp-net-core-web-api

In the Blazor, you can give a link to the file https://docs.microsoft.com/en-us/aspnet/core/blazor/file-downloads ?view=aspnetcore-6.0

Please let me know if this worked for you so I may start migration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants