HttpClient gzip

[Mr0N]

Description

HttpClient does not unpack gzip data from the HTTP response, even though the server includes the Content-Encoding: gzip header.

The GetStringAsync method does not work correctly in this context.

https://github.com/Mr0N/TestHttpClient/blob/master/TestHttpClient/Program.cs

info: Microsoft.Hosting.Lifetime[14]
      Now listening on: http://localhost:5258

Server:Content-Type:text/plain; charset=utf-8
Server:Date:Fri, 31 Jan 2025 18:16:24 GMT
Server:Server:Kestrel
Server:Content-Encoding:gzip
Server:Transfer-Encoding:chunked
Server:Vary:Accept-Encoding
▼♦
242615♦◄))))??♥.TN?♫
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\Users\user\source\repos\TestHttpClient\TestHttpClient

Reproduction Steps

using Microsoft.AspNetCore.ResponseCompression;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddResponseCompression(options =>
{
    options.Providers.Add<GzipCompressionProvider>();
});

var app = builder.Build();

app.Use(async (a,q) =>
{
    await q.Invoke();
    var select = a.Response.Headers
                .Select(a => "Server:"+a.Key + ":" + a.Value);
    Console.WriteLine(string.Join("\n\r",select));
});
app.UseResponseCompression();

app.MapGet("/check", () =>
{
    return "1234512345dddd";
});


Task.Run(async () =>
{
    await Task.Delay(4000);
    var handler = new HttpClientHandler
    {
       // AutomaticDecompression = System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate
    };
    var client = new HttpClient(handler);
    client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip");
    var response = await client.GetStringAsync("http://localhost:5258/check");
    Console.WriteLine(response);
});

app.Run();

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

No response

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[elgonzo]

// AutomaticDecompression = System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate

No real surprise there, as you commented this line out.

Pay attention to the documentation for the AutomaticDecompression setting (https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.automaticdecompression):

For the .NET Framework 4.x System.Net.Http binary in the Global Assembly Cache (GAC), the default value is None.

When the System.Net.Http NuGet package v4.1.0 to v4.3.2 is used, the default is GZip and Deflate.

After NuGet package v4.3.2, the default value of None is used.

Essentially, the default value for AutomaticDecompression is None unless you force your projects to use one of the outdated System.Net.Http packages from v4.1.0 to v4.3.2.

[Mr0N]

// AutomaticDecompression = System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate

No real surprise there, as you commented this line out.

Pay attention to the documentation for the AutomaticDecompression setting (https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.automaticdecompression):

For the .NET Framework 4.x System.Net.Http binary in the Global Assembly Cache (GAC), the default value is None.
When the System.Net.Http NuGet package v4.1.0 to v4.3.2 is used, the default is GZip and Deflate.
After NuGet package v4.3.2, the default value of None is used.

Essentially, the default value for AutomaticDecompression is None unless you force your projects to use one of the outdated System.Net.Http packages from v4.1.0 to v4.3.2.

Well, the server sends a header indicating that the content is encoded, so normally, the client should decode it. Yes, for the GetAsync method, this is not mandatory, but for the GetStringAsync method, which returns text, it would be preferable to do so, as otherwise, the client would not be working correctly.

[Mr0N]

And the server could theoretically send a compressed version even if it's not specified that the client supports it in the Accept-Encoding header.

[elgonzo]

Normally, HttpClient should behave as documented. Ignore the documentation at your own peril.
Whether you agree or disagree with the documented behavior of the API is a matter of discussion and opinion and does not make an issue nor a bug. This report should therefore be relocated to https://github.com/dotnet/runtime/discussions.

[wfurt]

This was discussed in #108163. Automatic decompression can lead to resource and security perils. If decompression is really what you want you should claim it explicitly @Mr0N.

[Mr0N]

This was discussed in #108163. Automatic decompression can lead to resource and security perils. If decompression is really what you want you should claim it explicitly @Mr0N.

I don't like that in this case the code works validly, as if nothing happened, but in reality, it works invalidly. Instead of normal text, it produces a bunch of gibberish. The client should probably throw an error or something like that because it processed invalidly and is ignoring it.