Implement crash reporting for iOS

[abhaysood]

Summary

This issue tracks changes needed across all systems for supporting crash reporting for iOS.

Documentation

• Update API schema for exceptions to support iOS crash reporting #1118

iOS

• Collect crashes using PLCrashReporter and store the raw crash report in a file #1130

Backend

Schema changes

• Add device_cpu_arch to event attributes. Max string length 32.
• Add following fields to exception:
  • signal — max string length 10
  • thread_name — max string length 64
  • thread_sequence — integer
  • os_build_number — max string length 12
• Add the following to the thread object:
  • sequence — integer
• Add the following fields to the frame object:
  • binary_name — string with max length 256
  • symbol_address — string representing a 16 bit hexadecimal number, max length 16
  • offset — number uint8
  • binary_address — string representing a 16 bit hexadecimal number, max length 16
  • in_app — boolean

Crash grouping

Algorithm

Exceptions are grouped together if all of the following conditions are met:

• They have the same signal.
• They have the same exception type.
• They have the same method name - consider first in-app frame of the exception, if in-app frame isn't available, use the first frame.
• The have the same file name - consider first in-app frame of the exception, if in-app frame isn't available, use the first frame. Do not consider line number.

Crash group name

Get the first frame with in_app=true in exception.exceptions.frames and create the title using the following format. If none in_app=true exist, then fallback to first frame.

file_name: method_name

Crash group description

When exception type and exception message are available:
Get the first object in exception.exceptions array and create the description using:

type: message

When exception type and exception message aren't available, then fallback to signal:

signal: message

Crash title (on crash detail page)

Same as "crash group name".

Symbolication

System requirements

• llvm-nm: to find the base image address as stored in the dSYM file.
• llvm-addr2line or llvm-symbolizer: : to map symbol addresses to specific lines of code.
• llvm-otool: to get the linker address of the binary.
• swift-demangle: to convert mangled Swift symbol names to a human-readable form.

Understanding the stacktrace

0   CoreFoundation                      0x00007ff8004cd561 0x7ff80039c000 + 1250657

• CoreFoundation is the binary name
• 0x00007ff8004cd561 is a symbol address.
• 0x7ff80039c000 is a binary image load address.
• 1250657 is symbol’s offset within a binary image.

Symbolication scripts

To perform symbolication, custom scripts have been created. Check this to understand how to use these scripts.

To symbolicate the app symbols, the below datapoints are needed

• DSYM file
• offsets
• CPU architecture (arm64e, x86_64, arm64, armv7k, armv6, armv7, armv7f, armv7s, arm, i386, arm64)

sh symbolicateWithDsym.sh MeasureDemo.app.dSYM/Contents/Resources/DWARF/MeasureDemo 3667246860 47120

Output
ViewController.tableView(_:didSelectRowAt:)
/Users/edpu/Documents/Workspace/measure/ios/MeasureDemo/MeasureDemo/ViewController.swift:52

To symbolicate the system symbols(UIKitCore, CoreFoundation etc), the below datapoints are needed

• Binary name
• Binary address
• Linker address
• Address to symbolicate
• CPU architecture (arm64e, x86_64, arm64, armv7k, armv6, armv7, armv7f, armv7s, arm, i386, arm64)
• Operating System Build (eg. 21E219) to figure out which system binary version is to be used

sh symbolicateSystemSymbols.sh UIKitCore 19d93e000

Output
-[UITableView _selectRowAtIndexPath:animated:scrollPosition:notifyDelegate:isCellMultiSelect:deselectPrevious:performCustomSelectionAction:]

Check out this doc to know more about iOS symbolication.

Dashboard

• The stacktrace for iOS should always be sorted by thread number.
• The crashed thread must be expanded by default, and other threads must be collapsed. See the following expected crash report.
• Add app_build along with app version to crash details page.
• Add os_version to crash details page.

Stacktrace sample

Unsymbolicated

Thread 0 Crashed:
0   CoreFoundation                      0x00007ff8004cd561 0x7ff80039c000 + 1250657
1   libobjc.A.dylib                     0x00007ff8000837e8 0x7ff80005c000 + 161768
2   CoreFoundation                      0x00007ff80039dc1f 0x7ff80039c000 + 7199
3   MeasureDemo                         0x0000000109007e2a 0x109005000 + 11818
4   MeasureDemo                         0x0000000109007323 0x109005000 + 8995
5   MeasureDemo                         0x00000001090073c1 0x109005000 + 9153
6   UIKitCore                           0x00007ff805faed4d 0x7ff804cbc000 + 19869005
7   UIKitCore                           0x00007ff805faf13d 0x7ff804cbc000 + 19870013
8   UIKitCore                           0x00007ff805faf24f 0x7ff804cbc000 + 19870287
9   UIKitCore                           0x00007ff806346231 0x7ff804cbc000 + 23634481
10  UIKitCore                           0x00007ff806346717 0x7ff804cbc000 + 23635735
11  libdispatch.dylib                   0x00007ff8001783ec 0x7ff800176000 + 9196
12  libdispatch.dylib                   0x00007ff8001796d8 0x7ff800176000 + 14040
13  libdispatch.dylib                   0x00007ff80018848c 0x7ff800176000 + 74892
14  libdispatch.dylib                   0x00007ff800187ef2 0x7ff800176000 + 73458
15  CoreFoundation                      0x00007ff800429b34 0x7ff80039c000 + 580404
16  CoreFoundation                      0x00007ff80042446f 0x7ff80039c000 + 558191
17  CoreFoundation                      0x00007ff8004236ed 0x7ff80039c000 + 554733
18  GraphicsServices                    0x00007ff8103ba08f 0x7ff8103b7000 + 12431
19  UIKitCore                           0x00007ff805cdf6ee 0x7ff804cbc000 + 16922350
20  UIKitCore                           0x00007ff805ce416e 0x7ff804cbc000 + 16941422
21  UIKitCore                           0x00007ff804d5fb73 0x7ff804cbc000 + 670579
22  MeasureDemo                         0x000000010900a02b 0x109005000 + 20523
23  MeasureDemo                         0x0000000109009fa7 0x109005000 + 20391
24  MeasureDemo                         0x000000010900a098 0x109005000 + 20632
25  ???                                 0x00000001090303e0 0x0 + 0
26  ???                                 0x0000000114246366 0x0 + 0

Thread 1:
0   libsystem_kernel.dylib              0x0000000109544af6 0x109542000 + 10998
1   libsystem_pthread.dylib             0x00000001090ddbdd 0x1090db000 + 11229
2   libsystem_pthread.dylib             0x00000001090dcacf 0x1090db000 + 6863

Thread 2:
0   libsystem_kernel.dylib              0x0000000109544af6 0x109542000 + 10998
1   libsystem_pthread.dylib             0x00000001090ddbdd 0x1090db000 + 11229
2   libsystem_pthread.dylib             0x00000001090dcacf 0x1090db000 + 6863

Thread 3:
0   libsystem_kernel.dylib              0x00000001095433ce 0x109542000 + 5070
1   libsystem_kernel.dylib              0x0000000109551c88 0x109542000 + 64648
2   libsystem_kernel.dylib              0x000000010954a510 0x109542000 + 34064
3   libsystem_kernel.dylib              0x00000001095436bd 0x109542000 + 5821
4   CoreFoundation                      0x00007ff8004298a3 0x7ff80039c000 + 579747
5   CoreFoundation                      0x00007ff80042402b 0x7ff80039c000 + 557099
6   CoreFoundation                      0x00007ff8004236ed 0x7ff80039c000 + 554733
7   Foundation                          0x00007ff800ea8c9a 0x7ff8007a5000 + 7355546
8   Foundation                          0x00007ff800ea8f18 0x7ff8007a5000 + 7356184
9   UIKitCore                           0x00007ff805dbc7aa 0x7ff804cbc000 + 17827754
10  Foundation                          0x00007ff800ed34d5 0x7ff8007a5000 + 7529685
11  libsystem_pthread.dylib             0x00000001090e118b 0x1090db000 + 24971
12  libsystem_pthread.dylib             0x00000001090dcae3 0x1090db000 + 6883

Thread 4:
0   libsystem_kernel.dylib              0x0000000109544af6 0x109542000 + 10998
1   libsystem_pthread.dylib             0x00000001090ddbdd 0x1090db000 + 11229
2   libsystem_pthread.dylib             0x00000001090dcacf 0x1090db000 + 6863

Symbolicated

Thread 0 Crashed:
0   CoreFoundation                      0x00007ff8004cd561 __exceptionPreprocess + 226
1   libobjc.A.dylib                     0x00007ff8000837e8 objc_exception_throw + 48
2   CoreFoundation                      0x00007ff80039dc1f -[__NSArray0 objectEnumerator] + 0
3   MeasureDemo                         0x000000010215ee2a ViewController.triggerCrash(type:): 52
4   MeasureDemo                         0x000000010215e323 ViewController.tableView(_:didSelectRowAt:): 50
5   MeasureDemo                         0x000000010215e3c1 @objc ViewController.tableView(_:didSelectRowAt:): 139
6   UIKitCore                           0x00007ff805faed4d -[UITableView _selectRowAtIndexPath:animated:scrollPosition:notifyDelegate:isCellMultiSelect:deselectPrevious:performCustomSelectionAction:] + 2221
7   UIKitCore                           0x00007ff805faf13d -[UITableView _userSelectRowAtPendingSelectionIndexPath:animatedSelection:] + 384
8   UIKitCore                           0x00007ff805faf24f -[UITableView _userSelectRowAtPendingSelectionIndexPath:] + 247
9   UIKitCore                           0x00007ff806346231 -[_UIAfterCACommitBlock run] + 55
10  UIKitCore                           0x00007ff806346717 -[_UIAfterCACommitQueue flush] + 189
11  UIKitCore                           0x00007ff805ceeed2 _runAfterCACommitDeferredBlocks + 784
12  UIKitCore                           0x00007ff805cddd46 _cleanUpAfterCAFlushAndRunDeferredBlocks + 96
13  UIKitCore                           0x00007ff805d12a71 _afterCACommitHandler + 58
14  CoreFoundation                      0x00007ff800429078 __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
15  CoreFoundation                      0x00007ff800423a0e __CFRunLoopDoObservers + 534
16  CoreFoundation                      0x00007ff800423f56 __CFRunLoopRun + 1158
17  CoreFoundation                      0x00007ff8004236ed CFRunLoopRunSpecific + 557
18  GraphicsServices                    0x00007ff8103ba08f GSEventRunModal + 137
19  UIKitCore                           0x00007ff805cdf6ee -[UIApplication _run] + 972
20  UIKitCore                           0x00007ff805ce416e UIApplicationMain + 123
21  UIKitCore                           0x00007ff804d5fb73 __swift_destroy_boxed_opaque_existential_1Tm + 10307
22  MeasureDemo                         0x000000010216102b static UIApplicationDelegate.main() + 123
23  MeasureDemo                         0x0000000102160fa7 static AppDelegate.$main() + 39
24  MeasureDemo                         0x0000000102161098 main + 24
25  ???                                 0x00000001026023e0 0x0 + 0
26  ???                                 0x000000010ad09366 0x0 + 0

Thread 1:
0   libsystem_kernel.dylib              0x0000000102694af6 __workq_kernreturn + 10
1   libsystem_pthread.dylib             0x00000001027d8bdd _pthread_wqthread + 416
2   libsystem_pthread.dylib             0x00000001027d7acf start_wqthread + 15

Thread 2:
0   libsystem_kernel.dylib              0x00000001026933ce mach_msg2_trap + 10
1   libsystem_kernel.dylib              0x00000001026a1c88 mach_msg2_internal + 84
2   libsystem_kernel.dylib              0x000000010269a510 mach_msg_overwrite + 653
3   libsystem_kernel.dylib              0x00000001026936bd mach_msg + 19
4   CoreFoundation                      0x00007ff8004298a3 __CFRunLoopServiceMachPort + 143
5   CoreFoundation                      0x00007ff80042402b __CFRunLoopRun + 1371
6   CoreFoundation                      0x00007ff8004236ed CFRunLoopRunSpecific + 557
7   Foundation                          0x00007ff800ea8c9a -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 213
8   Foundation                          0x00007ff800ea8f18 -[NSRunLoop(NSRunLoop) runUntilDate:] + 72
9   UIKitCore                           0x00007ff805dbc7aa -[UIEventFetcher threadMain] + 518
10  Foundation                          0x00007ff800ed34d5 __NSThread__start__ + 1024
11  libsystem_pthread.dylib             0x00000001027dc18b _pthread_start + 99
12  libsystem_pthread.dylib             0x00000001027d7ae3 thread_start + 15

Thread 3:
0   libsystem_kernel.dylib              0x0000000102694af6 __workq_kernreturn + 10
1   libsystem_pthread.dylib             0x00000001027d8bdd _pthread_wqthread + 416
2   libsystem_pthread.dylib             0x00000001027d7acf start_wqthread + 15

Thread 4:
0   libsystem_kernel.dylib              0x0000000102694af6 __workq_kernreturn + 10
1   libsystem_pthread.dylib             0x00000001027d8bdd _pthread_wqthread + 416
2   libsystem_pthread.dylib             0x00000001027d7acf start_wqthread + 15

Android

• Add thread name to exception #1127
• Add CPU architecture to attributes #1126