StackOverFlowException no longer breaks into the debugger on F5, instead the app hangs

[Olina-Zhang]

.NET version

.NET 10.0 SDK build: 10.0.100-preview.4.25177.19

Did it work in .NET Framework?

Yes

Did it work in any of the earlier releases of .NET Core or .NET 5+?

Yes, cannot repro in previous .NET versions, regressed from .NET 10.0 Preview 4, cannot repro in .NET 10.0 Preview 3.

Issue description

Simpler repro:

   public static void Main()
   {
       CauseStackOverflow();
   }
 
   private static void CauseStackOverflow()
   {
       CauseStackOverflow();
   } 

After the update VS hangs

---

Old repro:

StackOverFlowException cannot be popped up when errorProvider with a large size icon: 256*256

.NET 10: when running app, cannot get StackOverFlowException and runtime form

ErrorProvider_NET10.mp4

.NET 9 and .NET framework 4.8.1: get StackOverFlowException when running app

System.StackOverflowException
HResult=0x800703E9
Source=
StackTrace:

ErrorProvider_NET9.mp4

Steps to reproduce

Repro steps:

1. Create a Winforms .NET 10 app with TextBox and ErrorProvider added
2. Give a large icon with 256*256 size for errorProvider's Icon property
3. Double-click form designer, SetError for textBox in Form1_Load event

private void Form1_Load(object sender, EventArgs e)
{
 errorProvider1.SetError(textBox1, "testing");
}

4. Build and run application

or use this sample app: WinFormsApp31.zip

[LeafShi1]

This issue appears from PR dotnet/winforms#13169

The exception pop up from InitializeFromGdiPlus

The call stack:

System.Private.Windows.GdiPlus.dll!Windows.Win32.Graphics.GdiPlus.GpRegionExtensions.InitializeFromGdiPlus(Windows.Win32.Graphics.GdiPlus.GpRegion* region, Windows.Win32.Graphics.GdiPlus.GpGraphics* graphics) Line 34	C#
System.Private.Windows.GdiPlus.dll!Windows.Win32.Graphics.GdiPlus.GpRegionExtensions.GetRegionScope(Windows.Win32.Foundation.IPointer<Windows.Win32.Graphics.GdiPlus.GpRegion> region, Windows.Win32.Foundation.IPointer<Windows.Win32.Graphics.GdiPlus.GpGraphics> graphics) Line 13	C#
System.Windows.Forms.dll!System.Windows.Forms.ErrorProvider.ErrorWindow.Update(bool timerCaused) Line 377	C#
System.Windows.Forms.dll!System.Windows.Forms.ErrorProvider.ErrorWindow.Add(System.Windows.Forms.ErrorProvider.ControlItem item) Line 60	C#
System.Windows.Forms.dll!System.Windows.Forms.ErrorProvider.ControlItem.AddToWindow() Line 275	C#
System.Windows.Forms.dll!System.Windows.Forms.ErrorProvider.ControlItem.Error.set(string value) Line 131	C#
System.Windows.Forms.dll!System.Windows.Forms.ErrorProvider.SetError(System.Windows.Forms.Control control, string value) Line 792	C#
ScratchProject.dll!ScratchProject.Form1.Form1_Load(object sender, System.EventArgs e) Line 19	C#

[Olina-Zhang]

It appears to be a regression from the .NET 10.0 Preview 4 SDK version, which could not be reproduced in the .NET 10.0 Preview 3 test-pass version: 10.0.100-preview.3.25174.5

[Tanya-Solyanik]

@Olina-Zhang - thank you for the detailed bug and the investigation. Did you find out why we expect this stack overflow?
Do you think you could create a unit test(in the UI integration tests assembly) that shows this error? Create the icon programmatically, for example ask copilot "create a System.Drawing.Icon sized 256x256", and then set ErrorProvider1.Icon in the unit test and show the parent form.

[Olina-Zhang]

Did you find out why we expect this stack overflow?

https://learn.microsoft.com/en-us/dotnet/api/system.windows.forms.errorprovider.icon?view=windowsdesktop-9.0, I didn't find any doc about expecting StackOverFlowException, tested .NET framework 3.1~4.8.1 and .NET version, they are same. But recently, behavior is changed.

[Olina-Zhang]

Do you think you could create a unit test(in the UI integration tests assembly) that shows this error?

[WinFormsFact]
public async Task ErrorProvider_SetIcon_ShowsIconOnForm()
{
    await RunFormAsync(
        () =>
        {
            Form form = new();
            TextBox textBox = new() { Location = new Point(10, 10) };
            form.Controls.Add(textBox);

            // Create a 256x256 icon for testing
            using Bitmap bitmap = new(256, 256);
            using Graphics graphics = Graphics.FromImage(bitmap);
            graphics.FillRectangle(Brushes.Red, 0, 0, 256, 256);
            Icon icon = Icon.FromHandle(bitmap.GetHicon());

            ErrorProvider errorProvider = new()
            {
                Icon = icon
            };
            errorProvider.SetError(textBox, "Test error");

            return (form, textBox);
        },
        async (form, textBox) =>
        {
            form.Show();
            await Task.Delay(1000); // Wait for the form to render

            // Verify the icon is set correctly
            ErrorProvider errorProvider = (ErrorProvider)form.Controls[0].Tag;
            Assert.NotNull(errorProvider.Icon);
            Assert.Equal(256, errorProvider.Icon.Width);
            Assert.Equal(256, errorProvider.Icon.Height);
        });
}

In created UI integration test, when debugging it, it shows that error directly, when running it, shows it in Output window

Debug test:

Run test:

StackOverFlow_ErrorProvider.mp4

[Tanya-Solyanik]

Call Stack - https://github.com/Tanya-Solyanik/winforms/tree/stack-overflow-before
GpRegion::CreateDeviceRegion Line 1213 C++ Symbols loaded.
GpRegion::CreateDeviceRegion Line 1226 C++ Symbols loaded.
GpRegion::UpdateDeviceRegion Line 1303 C++ Symbols loaded.
[Inline Frame] GpRegion::IsInfinite Line 1881 C++ Symbols loaded.
GdipIsInfiniteRegion Line 3058 C++ Symbols loaded.
[Managed to Native Transition] Annotated Frame
Windows.Win32.Graphics.GdiPlus.GpRegionExtensions.InitializeFromGdiPlus Line 34 C# Symbols loaded.
Windows.Win32.Graphics.GdiPlus.GpRegionExtensions.GetRegionScope Line 13 C# Symbols loaded.
System.Windows.Forms.ErrorProvider.ErrorWindow.Update Line 377 C# Symbols loaded.
System.Windows.Forms.ErrorProvider.ErrorWindow.Add Line 60 C# Symbols loaded.
System.Windows.Forms.ErrorProvider.ControlItem.AddToWindow Line 275 C# Symbols loaded.

[Tanya-Solyanik]

I still see Stack Overflow exception reported in the VS output window,
Exception thrown at 0x00007FF9F9B1569B (GdiPlus.dll) in ScratchProject.exe: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x000000A181205FC8).

and the application hangs. Before the SDK update, we would report the stack overflow:
ZwWaitForSingleObject Line 268 Unknown Symbols loaded.
WaitForSingleObjectEx Line 1328 C Symbols loaded.
RtlpCallVectoredHandlers Line 204 C Symbols loaded.
[Inline Frame] RtlCallVectoredExceptionHandlers Line 358 C Symbols loaded.
RtlDispatchException Line 413 C Symbols loaded.
KiUserExceptionDispatch Line 771 Unknown Symbols loaded.
GpRegion::CreateDeviceRegion Line 1210 C++ Symbols loaded.