Flaky test failure: TTID Tracker

[vaind]

Description

Noticed this in CI:

 Failed to load "/home/runner/work/sentry-dart/sentry-dart/flutter/example/integration_test/all.dart": type 'NoOpSentrySpan' is not a subtype of type 'SentryTracer' in type cast
  package:sentry_flutter/src/navigation/time_to_initial_display_tracker.dart 54:32  TimeToInitialDisplayTracker.track
  package:sentry_flutter/src/navigation/time_to_display_tracker.dart 29:24          TimeToDisplayTracker.track
  package:sentry_flutter/src/integrations/native_app_start_handler.dart 46:40       NativeAppStartHandler.call
  ===== asynchronous gap ===========================
  package:sentry_flutter/src/integrations/native_app_start_integration.dart 59:11   NativeAppStartIntegration.call.timingsCallback

[buenaflor]

this is fixed in 8.11.1, if you merge main it shouldn't fail anymore

[vaind]

The same error is still printed if I debug integration_test.dart (because I'm trying to figure out why it hangs on my PR):

[sentry] [warning] FramesTrackingIntegration disabled: incompatible binding, SentryFlutterWidgetsBinding has not been instantiated. Please, use SentryFlutterWidgetsBinding.ensureInitialized() instead of FlutterWidgetsBinding.ensureInitialized()
[sentry] [debug] release: [email protected]+1
[sentry] [debug] ReplayRecorder #1: starting capture (352.0x768.0 @ 1 Hz).
[sentry] [warning] TTFD tracker not started or already completed. Dropping TTFD measurement.
[sentry] [warning] ReplayRecorder #1: SentryScreenshotWidget is not attached, skipping capture.
[sentry] [debug] ReplayRecorder #1: stopping capture.
[sentry] [error] Error while capturing native app start
[sentry] _TypeError (type 'NoOpSentrySpan' is not a subtype of type 'SentryTracer' in type cast)
[sentry] #0      TimeToFullDisplayTracker.track (package:sentry_flutter/src/navigation/time_to_full_display_tracker.dart:46:32)
         #1      TimeToDisplayTracker.track (package:sentry_flutter/src/navigation/time_to_display_tracker.dart:38:26)
time_to_display_tracker.dart:38
         <asynchronous suspension>
         #2      NativeAppStartHandler.call (package:sentry_flutter/src/integrations/native_app_start_handler.dart:46:5)
native_app_start_handler.dart:46
         <asynchronous suspension>
         #3      NativeAppStartIntegration.call.timingsCallback (package:sentry_flutter/src/integrations/native_app_start_integration.dart:59:11)
native_app_start_integration.dart:59
         <asynchronous suspension>
[sentry] [debug] ReplayRecorder #1: capture stopped.
[sentry] [debug] Metrics: nothing to flush
✓ setup sentry and render app
[sentry] [debug] ReplayRecorder #1: stopping capture.
[sentry] [debug] ReplayRecorder #1: capture stopped.

Maybe the app is already closed/closing (or rather Sentry SDK is) by the time the tracker triggers?

[buenaflor]

yeah that's the issue since it runs async, but not sure why this is still popping up

[buenaflor]

I merged the 8.11.1 release branch into main and the deps are still on 8.12.0-beta.1 which doesnt include those changes from 8.11.1 maybe we need to run another release?

nvm there is still one left, I guess we can also guard that and that should be it

[vaind]

Still flaky

❌ /home/runner/work/sentry-dart/sentry-dart/flutter/test/navigation/sentry_display_widget_test.dart: SentryDisplayWidget reports manual ttid span after didPush (failed)
  ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
  The following _TypeError was thrown running a test:
  type 'Null' is not a subtype of type 'SentryTracer' in type cast
  
  When the exception was thrown, this was the stack:
  #0      main.<anonymous closure> (file:///home/runner/work/sentry-dart/sentry-dart/flutter/test/navigation/sentry_display_widget_test.dart:35:42)
  <asynchronous suspension>
  #1      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:189:15)
  <asynchronous suspension>
  #2      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1027:5)
  <asynchronous suspension>
  <asynchronous suspension>
  (elided one frame from package:stack_trace)
  
  The test description was:
    SentryDisplayWidget reports manual ttid span after didPush
  ════════════════════════════════════════════════════════════════════════════════════════════════════
  Warning: At least one test in this suite creates an HttpClient. When running a test suite that uses
  TestWidgetsFlutterBinding, all HTTP requests will return status code 400, and no network request
  will actually be made. Any test expecting a real network connection and status code will fail.
  To test code that needs an HttpClient, provide your own HttpClient implementation to the code under
  test, so that your test can consistently provide a testable response to the code under test.
  Test failed. See exception logs above.
  The test description was: SentryDisplayWidget reports manual ttid span after didPush
  

[buenaflor]

I'll have a look

[hantrungkien]

@buenaflor We are using 8.11.2 and get a error when app start. Please help me.

[sentry] [warning] FramesTrackingIntegration disabled: incompatible binding, SentryFlutterWidgetsBinding has not been instantiated. Please, use SentryFlutterWidgetsBinding.ensureInitialized() instead of FlutterWidgetsBinding.ensureInitialized()

[sentry] [error] Error while capturing native app start
         type 'NoOpSentrySpan' is not a subtype of type 'SentryTracer' in type cast
         #0      TimeToFullDisplayTracker.track (package:sentry_flutter/src/navigation/time_to_full_display_tracker.dart:46:32)
         #1      TimeToDisplayTracker.track (package:sentry_flutter/src/navigation/time_to_display_tracker.dart:38:26)
         <asynchronous suspension>
         #2      NativeAppStartHandler.call (package:sentry_flutter/src/integrations/native_app_start_handler.dart:46:5)
         <asynchronous suspension>
         #3      NativeAppStartIntegration.call.timingsCallback (package:sentry_flutter/src/integrations/native_app_start_integration.dart:59:11)
         <asynchronous suspension>

[vaind]

@hantrungkien that's been fixed in #2508. You can pick up the latest v8.12 beta or wait for the stable release

[hantrungkien]

@vaind I got it but when using v8.12-beta2 I still get a warning message

[sentry] [warning] FramesTrackingIntegration disabled: incompatible binding, SentryFlutterWidgetsBinding has not been instantiated. Please, use SentryFlutterWidgetsBinding.ensureInitialized() instead of FlutterWidgetsBinding.ensureInitialized()

[vaind]

I meant the error is fixed. @buenaflor could know about the warning

[buenaflor]

@vaind I got it but when using v8.12-beta2 I still get a warning message

[sentry] [warning] FramesTrackingIntegration disabled: incompatible binding, SentryFlutterWidgetsBinding has not been instantiated. Please, use SentryFlutterWidgetsBinding.ensureInitialized() instead of FlutterWidgetsBinding.ensureInitialized()

To fix this you can do SentryFlutterWidgetsBinding.ensureInitialized() instead of FlutterWidgetsBinding.ensureInitialized() as the message suggests

it's also safe to ignore if you don't want to do that

[hantrungkien]

@buenaflor I'm using SentryWidgetsFlutterBinding.ensureInitialized(); but I still get this warning message.

[vaind]

@buenaflor I'm using SentryWidgetsFlutterBinding.ensureInitialized(); but I still get this warning message.

@hantrungkien The order of calls is important. As this is not related to "Test failure: TTID Tracker", could you please create a new issue if you still encounter the problem? That way you're more likely to get help becauase this issue is unrelated and will be closed once tests are no longer flaky.