Exceptions within Console/Display classes should probably be logged but swallowed - Currently they break test execution.

That is unfortunate, but imho the correct way to do this. Otherwise you would not see it and we have no other way of receiving that kind of information.

I've added 1000ms timeout because that seemed like lightyears in regex matching, especially on the source generated. I would start by wrapping the match into try catch, and report the string that is being matched.

Great that you are already consuning preview, so we can debug this with a simpler loop.

There should be new package in the preview channel in about half an hour, please update when convenient for you. Hopefully we will see what the source of the timeout is.

@nohwnd

  Error: Unhandled Exception: 
  System.Text.RegularExpressions.RegexMatchTimeoutException: Parsing stack trace 
  line with regex timed out. Input: '   at 
  System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xbe'
   ---> System.Text.RegularExpressions.RegexMatchTimeoutException: The Regex 
  engine has timed out while trying to match a pattern to an input string. This 
  can occur for many reasons, including very large inputs or excessive 
  backtracking caused by nested quantifiers, back-references and other factors.
     at 
  System.Text.RegularExpressions.RegexRunner.g__ThrowRegexTimeout|25
  _0() + 0x5f
     at 
  System.Text.RegularExpressions.Generated.F06D33C3F8C8C3FD257C1
  A1967E3A3BAC4BE9C8EC41CC9366C764C2205C68F0CE__GetFrameRegex_1.RunnerFactory.Runn
  er.TryMatchAtCurrentPosition(ReadOnlySpan`1) + 0xbbd
     at 
  System.Text.RegularExpressions.Generated.F06D33C3F8C8C3FD257C1
  A1967E3A3BAC4BE9C8EC41CC9366C764C2205C68F0CE__GetFrameRegex_1.RunnerFactory.Runn
  er.Scan(ReadOnlySpan`1) + 0x35
     at System.Text.RegularExpressions.Regex.ScanInternal(RegexRunnerMode, 
  Boolean, String, Int32, RegexRunner, ReadOnlySpan`1, Boolean) + 0x36
     at System.Text.RegularExpressions.Regex.RunSingleMatch(RegexRunnerMode, 
  Int32, String, Int32, Int32, Int32) + 0x157
     at System.Text.RegularExpressions.Regex.Match(String) + 0x3e
     at 
  Microsoft.Testing.Platform.OutputDevice.Terminal.TerminalTestReporter.AppendStac
  kFrame(ITerminal, String) + 0x115
     --- End of inner exception stack trace ---
     at 
  System.Text.RegularExpressions.RegexRunner.g__ThrowRegexTimeout|25
  _0() + 0x5f
     at 
  System.Text.RegularExpressions.Generated.F06D33C3F8C8C3FD257C1
  A1967E3A3BAC4BE9C8EC41CC9366C764C2205C68F0CE__GetFrameRegex_1.RunnerFactory.Runn
  er.TryMatchAtCurrentPosition(ReadOnlySpan`1) + 0xbbd
     at 
  System.Text.RegularExpressions.Generated.F06D33C3F8C8C3FD257C1
  A1967E3A3BAC4BE9C8EC41CC9366C764C2205C68F0CE__GetFrameRegex_1.RunnerFactory.Runn
  er.Scan(ReadOnlySpan`1) + 0x35
     at System.Text.RegularExpressions.Regex.ScanInternal(RegexRunnerMode, 
  Boolean, String, Int32, RegexRunner, ReadOnlySpan`1, Boolean) + 0x36
     at System.Text.RegularExpressions.Regex.RunSingleMatch(RegexRunnerMode, 
  Int32, String, Int32, Int32, Int32) + 0x157
     at System.Text.RegularExpressions.Regex.Match(String) + 0x3e
     at 
  Microsoft.Testing.Platform.OutputDevice.Terminal.TerminalTestReporter.AppendStac
  kFrame(ITerminal, String) + 0x115
  --- End of stack trace from previous location ---
     at 
  Microsoft.Testing.Platform.OutputDevice.Terminal.TerminalTestReporter.AppendStac
  kFrame(ITerminal, String) + 0x9e8
     at 
  Microsoft.Testing.Platform.OutputDevice.Terminal.TerminalTestReporter.FormatStac
  kTrace(ITerminal, FlatException[], Int32) + 0x11a
     at 
  Microsoft.Testing.Platform.OutputDevice.Terminal.TerminalTestReporter.RenderTest
  Completed(ITerminal, String, String, String, String, TestOutcome, TimeSpan, 
  FlatException[], String, String, String, String) + 0x783
     at 
  Microsoft.Testing.Platform.OutputDevice.Terminal.TerminalTestReporter.<>c__Displ
  ayClass27_0.b__0(ITerminal terminal) + 0x63
     at 
  Microsoft.Testing.Platform.OutputDevice.Terminal.TestProgressStateAwareTerminal.
  WriteToTerminal(Action`1) + 0x94
     at 
  Microsoft.Testing.Platform.OutputDevice.Terminal.TerminalTestReporter.TestComple
  ted(String, String, String, String, String, TestOutcome, TimeSpan, FlatException
  [], String, String, String, String) + 0x3d2
     at 
  Microsoft.Testing.Platform.OutputDevice.TerminalOutputDevice.d__57
  .MoveNext() + 0x407
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Messages.AsyncConsumerDataProcessor.d__
  12.MoveNext() + 0x2ca
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Messages.AsyncConsumerDataProcessor.d
  __14.MoveNext() + 0x1f0
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Messages.AsynchronousMessageBus.d__17
  .MoveNext() + 0x775
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Messages.MessageBusProxy.d__6.MoveNex
  t() + 0xaa
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Hosts.CommonTestHost.d__16
  .MoveNext() + 0xd2
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Hosts.CommonTestHost.d__12.MoveN
  ext() + 0x318
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Hosts.ConsoleTestHost.d__10.MoveNex
  t() + 0x867
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at 
  Microsoft.Testing.Platform.Hosts.ConsoleTestHost.d__10.MoveNex
  t() + 0x10b3
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Hosts.CommonTestHost.d__10.MoveNext(
  ) + 0x233
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at Microsoft.Testing.Platform.Hosts.CommonTestHost.d__8.MoveNext() 
  + 0x18e
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at Microsoft.Testing.Platform.Hosts.CommonTestHost.d__8.MoveNext() 
  + 0x722
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at 
  Microsoft.Testing.Platform.Builder.TestApplication.d__17.MoveNext() + 
  0xaf
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at TestingPlatformEntryPoint.d__0.MoveNext() + 0x224
  --- End of stack trace from previous location ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 
  0xbe
     at 
  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotificat
  ion(Task, ConfigureAwaitOptions) + 0x4e
     at TestingPlatformEntryPoint.(String[] args) + 0x2e
     at TUnit!+0xed2b6c

outsider here, hoping to not distract -

How constrained does this mean? Is the system just totally trashed (swapping, etc.)?

I guess due to constrained system resources this is being hit.

It skips a test after 6 seconds and then takes over 3 minutes to pass one test. Is that expected?

  [+0/x0/?0] TUnit.Engine.Tests.dll (net8.0|arm64)(3s)
  [+0/x0/?0] TUnit.Engine.Tests.dll (net8.0|arm64)(6s)
  Skipping JsonOutputTests.Test...
  skipped Test (0ms)
  [+0/x0/?1] TUnit.Engine.Tests.dll (net8.0|arm64)(9s)
...
  [+0/x0/?1] TUnit.Engine.Tests.dll (net8.0|arm64)(3m 27s) 
  [+1/x0/?1] TUnit.Engine.Tests.dll (net8.0|arm64)(3m 30s)

Yeah I was looking at how long the tests run as well, but there was no complain from OP about the testing being slow, and I don't know their tests.

I've discussed the timeout issue internally, and the implementation of the timeout is not super helpful, because it uses wallclock time rather than how much time the regex actually spent executing on cpu. So if the thread / runner gets suspended, the regex will still time out. In all our tries we could not get the regex itself to run over 1ms, which is still more than I would expect on matching a single line. And the regex is linear, so it should be okay, to simply remove the timeout.

Removing the timeout sounds sensible to me