ci: fix the 32 byte inaccuracy of SDK size reports

Part of: https://linear.app/customerio/issue/MBL-155/track-the-binary-size-of-our-ios-sdk-upon-every-release

As called out in the ticket, the SDK size report comparing main branch to open PRs is inaccurate by 32 bytes. This commit fixes this so if you open a PR and not modify any code in it, the diff should show 0 bytes difference.

Testing consideration:
* The PR should see SDK size reports generated as before. QA sample app builds created and uploaded as before.
* We cannot fully test this until we merge into main and the main branch creates a new cache.

Notes for reviewer:
* I fixed this issue by setting up the CI to compile a version of the APN sample app that does not modify the Version.swift file. This compiled app is used to generate SDK size report. The CI continues to compile sample apps for QA testing and it should work as previously expected.
* Because the generate-sdk-size-report action is now compiling sample apps, I created a new action, build-sample-app, to make building a sample app re-usable.
* "fastlane ios build" command now takes optional arguments to configure it. QA sample apps will make builds differently then builds for generating SDK size reports.

commit-id:eddb8612

.github/actions/build-sample-app/action.yml

@@ -0,0 +1,123 @@
+name: Build one of the sample apps
+description: Encapsulate all the logic needed to compile a sample app. 
+
+inputs:
+  # If you want to save a new build, pass in this input. If you want to get the latest build, leave this input out.
+  apn-or-fcm:
+    description: 'Defines which push service to build the sample app for. Either "APN" or "FCM".'
+    type: string
+    required: true
+  sample-app:
+    description: 'The name of the sample app to build. Provide name of 1 of the directories in the Apps/ directory. Example: "CocoaPods-FCM"'
+    type: string
+    required: true
+  # The action we use to interact with Fastlane requires a JSON object for arguments, https://github.com/maierj/fastlane-action?tab=readme-ov-file#inputs
+  fastlane-build-args:
+    description: 'Arguments to pass to the Fastlane build command. Example: "{ "option1": "value1", "option2": "value2" }"'
+    type: string
+    required: false
+    default: '{}'
+  customerio-workspace-siteid:
+    description: 'The site ID for a workspace in Customer.io to set as default in compiled app, probably a secret.'
+    type: string
+    required: true 
+  customerio-workspace-cdp-api-key:
+    description: 'The CDP API key for a workspace in Customer.io to set as default in compiled app, probably a secret.'
+    type: string
+    required: true
+  GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64:
+    description: 'Maps to the secret, GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64. Used for code signing. See the Fastlane config files to learn more.'
+    type: string
+    required: true
+  FIREBASE_APP_DISTRIBUTION_SERVICE_ACCOUNT_CREDS_B64:
+    description: 'Set this input, only if you want to upload app for internal testing. Maps to the secret, FIREBASE_APP_DISTRIBUTION_SERVICE_ACCOUNT_CREDS_B64. Used for uploading compiled apps for testing. See the Fastlane config files to learn more.'
+    type: string
+    required: false
+    default: ''
+
+outputs:
+  app-xcarchive-path:
+    description: 'The full relative path to the xcarchive for the built sample app. Example: "Apps/APN-UIKit/build/App.xcarchive"'
+    value: ${{ steps.set-action-outputs.outputs.app-xcarchive-path }}
+  app-xcarchive-name:
+    description: 'The name of the xcarchive for the sample app build. Example: "App.xcarchive"'
+    value: ${{ steps.set-action-outputs.outputs.app-xcarchive-name }}
+
+runs:
+  using: "composite"
+  steps:
+    - uses: ./.github/actions/setup-ios
+
+    - name: Install CLI tools used in CI script 
+      shell: bash
+      run: |
+        brew install sd # used in CI script as an easier to use sed CLI. Replaces text in files. 
+        brew install xcbeautify # used by fastlane for output 
+    
+    - name: Install tools from Gemfile (ruby language) used for building our apps with 
+      uses: ruby/setup-ruby@v1
+      with:
+        ruby-version: '3.0'
+        bundler-cache: true # cache tools to make builds faster in future 
+
+    - name: Setup build environment to prepare for building
+      shell: bash
+      run: |
+        make setup_sample_app app=${{ inputs.sample-app }}
+        sd CUSTOMERIO_WORKSPACE_SITE_ID ${{ inputs.customerio-workspace-siteid }} "Apps/${{ inputs.sample-app }}/BuildEnvironment.swift"
+        sd CUSTOMERIO_WORKSPACE_CDP_API_KEY ${{ inputs.customerio-workspace-cdp-api-key }} "Apps/${{ inputs.sample-app }}/BuildEnvironment.swift"
+
+    - name: Does ${{ inputs.sample-app }} use CocoaPods? 
+      id: check_podfile_exists
+      uses: andstor/file-existence-action@v3
+      with:
+        files: "Apps/${{ inputs.sample-app }}/Podfile"
+
+    - name: Cache CocoaPods downloaded dependencies for faster builds in the future 
+      if: steps.check_podfile_exists.outputs.files_exists == 'true'
+      uses: actions/cache@v4
+      with:
+        path: Apps/${{ inputs.sample-app }}/Pods
+        key: ${{ runner.os }}-${{ inputs.sample-app}}-Pods-${{ github.ref }}
+        restore-keys: |
+          ${{ runner.os }}-${{ inputs.sample-app}}-Pods
+
+    - name: Cache SPM downloaded dependencies for faster builds in the future 
+      if: steps.check_podfile_exists.outputs.files_exists == 'false'
+      uses: actions/cache@v4
+      with:
+        path: Apps/${{ inputs.sample-app }}/spm_packages
+        key: ${{ runner.os }}-${{ inputs.sample-app}}-SPMPackages-${{ github.ref }}
+        restore-keys: |
+          ${{ runner.os }}-${{ inputs.sample-app}}-SPMPackages
+
+    - name: Run pod install if using CocoaPods 
+      if: steps.check_podfile_exists.outputs.files_exists == 'true'
+      shell: bash
+      run: make install_cocoapods_dependencies app=${{ inputs.sample-app }}
+
+    - name: Dump GitHub Action metadata because Fastlane uses it. Viewing it here helps debug JSON parsing code in Firebase. 
+      shell: bash
+      run: cat $GITHUB_EVENT_PATH
+
+    - name: Build app via Fastlane 
+      uses: maierj/[email protected]
+      with:
+        lane: "ios build"
+        subdirectory: "Apps/${{ inputs.sample-app }}"
+        options: ${{ inputs.fastlane-build-args }}
+      env: 
+        GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64: ${{ inputs.GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64 }}
+        FIREBASE_APP_DISTRIBUTION_SERVICE_ACCOUNT_CREDS_B64: ${{ inputs.FIREBASE_APP_DISTRIBUTION_SERVICE_ACCOUNT_CREDS_B64 }}
+
+    # xcodebuild creates builds that include a timestamp in the name. In order for bloaty to read the build, we need to rename it to a static name.
+    - name: Rename the same app build to a static name that we can generate SDK size reports with
+      shell: bash
+      run: mv Apps/${{ inputs.sample-app }}/build/*.xcarchive Apps/${{ inputs.sample-app }}/build/App.xcarchive 
+
+    - name: Set action output values
+      id: set-action-outputs
+      shell: bash
+      run: |
+        echo "app-xcarchive-path=$(echo Apps/${{ inputs.sample-app }}/build/App.xcarchive)" >> $GITHUB_OUTPUT
+        echo "app-xcarchive-name=$(echo App.xcarchive)" >> $GITHUB_OUTPUT

.github/actions/generate-sdk-size-report/action.yml

@@ -1,16 +1,12 @@
 name: Generate SDK size reports 
 # We only need to use 1 sample app build to generate the SDK size diff report. Therefore, we are hard-coding APN-UIKit into this action.
-description: Given a .xcarchive for the APN sample app, this action will generate all of the SDK size reports. 
+description: Run this action to generate all of the SDK size reports. 
 
-inputs:  
-  apn-app-xcarchive-name:
-    description: 'The name of the .xcarchive located inside of Apps/APN-UIKit/build/ to generate reports for. Example: "App.xcarchive"'
+inputs:
+  GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64:
+    description: 'Maps to the secret, GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64. Used for code signing. See the Fastlane config files to learn more.'
     type: string
-    required: true    
-  generate-main-diff-report:
-    description: 'Whether to generate the SDK size diff report between the main branch and the PR branch. If you are running on a PR, set this to true'
-    type: boolean
-    required: true 
+    required: true
 
 # 3 separate SDK size reports may be generated by this action. 
 outputs:
@@ -21,33 +17,48 @@ outputs:
     description: 'The SDK size report, as a string. This report includes dependencies as well as the SDK size.'
     value: ${{ steps.sdk-size-including-dependencies-report.outputs.stdout }}
   sdk-size-diff-report:
-    description: 'The SDK size report, as a string. This report is the size difference between passed in app and the last build on main branch. Only populated if generate-main-diff-report input variable is true.'
+    description: 'The SDK size report, as a string. This report is the size difference between the current branch code and the last build on main branch. Only populated if this action is run from a pull request.'
     value: ${{ steps.sdk-size-diff-report.outputs.stdout }}
 
 runs:
   using: "composite"
   steps:
-  - name: Set environment variables for convenience in future steps 
-    shell: bash
-    # Use each of the set variables in future steps like: ${{ env.WORKING_DIRECTORY }}    
-    run: echo "WORKING_DIRECTORY=Apps/APN-UIKit/build" >> $GITHUB_ENV
-
   - name: Assert running on macOS runner. Bloaty requires some complex compilation to run on Linux. Therefore, this action only supports macOS runners.
     shell: bash
     run: test $RUNNER_OS = 'macOS'
 
   - name: Install bloaty, in case it is not already 
     shell: bash
-    run: brew install bloaty || true # If it's already installed, this will fail without error
+    run: brew install bloaty || true # If it's already installed, this will fail without error    
+
+  - name: Make a build of the sample app to generate report for 
+    uses: ./.github/actions/build-sample-app
+    id: build-sample-app
+    with:
+      apn-or-fcm: 'APN'
+      sample-app: 'APN-UIKit'
+      # Pass in a hard-coded version and build number to ensure that all sample app builds that are compiled for SDK size reports are consistent.
+      # When we compare size reports between different builds, we want to ensure that the only difference is the SDK code that was modified in a PR. 
+      fastlane-build-args: '{"app_version": "1.0.0", "build_number": "1"}'
+      # workspace credentials do not matter since we are not using this app build. 
+      customerio-workspace-siteid: "12345"
+      customerio-workspace-cdp-api-key: "12345"      
+      GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64: ${{ inputs.GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64 }}
+
+  - name: Save the sample app build, if this is a main branch build 
+    if: github.ref == 'refs/heads/main'   
+    uses: ./.github/actions/main-sample-app-build 
+    with:
+      apn-app-xcarchive-name: ${{ steps.build-sample-app.outputs.app-xcarchive-name }}
 
   - name: Make the SDK size report
     uses: mathiasvr/[email protected]
     id: sdk-size-report
     with:
       run: |
         bloaty --source-filter ".*(customerio-ios\/Sources).*" \
-          -d compileunits --debug-file=${{ env.WORKING_DIRECTORY }}/${{ inputs.apn-app-xcarchive-name }}/dSYMs/APN\ UIKit.app.dSYM/Contents/Resources/DWARF/APN\ UIKit \
-          ${{ env.WORKING_DIRECTORY }}/${{ inputs.apn-app-xcarchive-name }}/Products/Applications/APN\ UIKit.app/APN\ UIKit \
+          -d compileunits --debug-file=${{ steps.build-sample-app.outputs.app-xcarchive-path }}/dSYMs/APN\ UIKit.app.dSYM/Contents/Resources/DWARF/APN\ UIKit \
+          ${{ steps.build-sample-app.outputs.app-xcarchive-path }}/Products/Applications/APN\ UIKit.app/APN\ UIKit \
           -n 0 
     
   - name: Make the SDK size report, including dependencies
@@ -56,15 +67,15 @@ runs:
     with: 
       run: |
         bloaty --source-filter ".*(customerio-ios\/Sources)|(SourcePackages\/checkouts).*" \
-          -d compileunits --debug-file=${{ env.WORKING_DIRECTORY }}/${{ inputs.apn-app-xcarchive-name }}/dSYMs/APN\ UIKit.app.dSYM/Contents/Resources/DWARF/APN\ UIKit \
-          ${{ env.WORKING_DIRECTORY }}/${{ inputs.apn-app-xcarchive-name }}/Products/Applications/APN\ UIKit.app/APN\ UIKit \
+          -d compileunits --debug-file=${{ steps.build-sample-app.outputs.app-xcarchive-path }}/dSYMs/APN\ UIKit.app.dSYM/Contents/Resources/DWARF/APN\ UIKit \
+          ${{ steps.build-sample-app.outputs.app-xcarchive-path }}/Products/Applications/APN\ UIKit.app/APN\ UIKit \
           -n 0 
   
   # We want to determine if the SDK's binary size will change after merging a PR. 
   # To do that, we take the latest sample app build from the main branch and compare it to the app passed into this action. 
 
   - name: Download latest main sample app build to generate SDK size diff report       
-    if: ${{ inputs.generate-main-diff-report }} == true
+    if: ${{ github.event_name == 'pull_request' }}
     id: download-latest-main-build
     uses: ./.github/actions/main-sample-app-build
 
@@ -76,5 +87,5 @@ runs:
       run: |
         bloaty --source-filter ".*(customerio-ios\/Sources).*" -d compileunits \
           --debug-file="${{ steps.download-latest-main-build.outputs.apn-app-xcarchive-path }}/dSYMs/APN UIKit.app.dSYM/Contents/Resources/DWARF/APN UIKit" \
-          --debug-file="${{ env.WORKING_DIRECTORY }}/${{ inputs.apn-app-xcarchive-name }}/dSYMs/APN UIKit.app.dSYM/Contents/Resources/DWARF/APN UIKit" \
-          "${{ env.WORKING_DIRECTORY }}/${{ inputs.apn-app-xcarchive-name }}/Products/Applications/APN UIKit.app/APN UIKit" -- "${{ steps.download-latest-main-build.outputs.apn-app-xcarchive-path }}/Products/Applications/APN UIKit.app/APN UIKit" 
+          --debug-file="${{ steps.build-sample-app.outputs.app-xcarchive-path }}/dSYMs/APN UIKit.app.dSYM/Contents/Resources/DWARF/APN UIKit" \
+          "${{ steps.build-sample-app.outputs.app-xcarchive-path }}/Products/Applications/APN UIKit.app/APN UIKit" -- "${{ steps.download-latest-main-build.outputs.apn-app-xcarchive-path }}/Products/Applications/APN UIKit.app/APN UIKit" 

.github/workflows/build-sample-apps.yml

@@ -66,82 +66,16 @@ jobs:
     name: Building app...${{ matrix.sample-app }}
     steps:
     - uses: actions/checkout@v4
-    - uses: ./.github/actions/setup-ios
 
-    - name: Install CLI tools used in CI script 
-      run: |
-        brew install sd # used in CI script as an easier to use sed CLI. Replaces text in files. 
-        brew install xcbeautify # used by fastlane for output 
-    
-    - name: Install tools from Gemfile (ruby language) used for building our apps with 
-      uses: ruby/setup-ruby@v1
-      with:
-        ruby-version: '3.0'
-        bundler-cache: true # cache tools to make builds faster in future 
-
-    - name: Setup APN build environment to prepare for building
-      if: ${{ matrix.apn-or-fcm == 'APN' }}
-      run: |
-        make setup_sample_app app=${{ matrix.sample-app }}
-        sd CUSTOMERIO_WORKSPACE_SITE_ID ${{ secrets.CUSTOMERIO_APN_WORKSPACE_SITE_ID }} "Apps/${{ matrix.sample-app }}/BuildEnvironment.swift"
-        sd CUSTOMERIO_WORKSPACE_CDP_API_KEY ${{ secrets.CUSTOMERIO_APN_WORKSPACE_CDP_API_KEY }} "Apps/${{ matrix.sample-app }}/BuildEnvironment.swift"
-
-    - name: Setup FCM build environment to prepare for building
-      if: ${{ matrix.apn-or-fcm == 'FCM' }}
-      run: |
-        make setup_sample_app app=${{ matrix.sample-app }}
-        sd CUSTOMERIO_WORKSPACE_SITE_ID ${{ secrets.CUSTOMERIO_FCM_WORKSPACE_SITE_ID }} "Apps/${{ matrix.sample-app }}/BuildEnvironment.swift"
-        sd CUSTOMERIO_WORKSPACE_CDP_API_KEY ${{ secrets.CUSTOMERIO_FCM_WORKSPACE_CDP_API_KEY }} "Apps/${{ matrix.sample-app }}/BuildEnvironment.swift"
-
-    - name: Does ${{ matrix.sample-app }} use CocoaPods? 
-      id: check_podfile_exists
-      uses: andstor/file-existence-action@v3
-      with:
-        files: "Apps/${{ matrix.sample-app }}/Podfile"
-
-    - name: Cache CocoaPods downloaded dependencies for faster builds in the future 
-      if: steps.check_podfile_exists.outputs.files_exists == 'true'
-      uses: actions/cache@v4
-      with:
-        path: "Apps/${{ matrix.sample-app }}/Pods"
-        key: ${{ runner.os }}-${{ matrix.sample-app}}-Pods
-        restore-keys: |
-          ${{ runner.os }}-${{ matrix.sample-app}}-Pods
-
-    - name: Run pod install if using CocoaPods 
-      if: steps.check_podfile_exists.outputs.files_exists == 'true'
-      run: make install_cocoapods_dependencies app=CocoaPods-FCM
-
-    - name: Dump GitHub Action metadata because Fastlane uses it. Viewing it here helps debug JSON parsing code in Firebase. 
-      run: cat $GITHUB_EVENT_PATH
-
-    - name: Build app via Fastlane 
-      uses: maierj/[email protected]
+    - name: Build and upload app build for QA testing 
+      uses: ./.github/actions/build-sample-app
       with:
-        lane: "ios build"
-        subdirectory: "Apps/${{ matrix.sample-app }}"
-      env: 
+        apn-or-fcm: ${{ matrix.apn-or-fcm }}
+        sample-app: ${{ matrix.sample-app }}
+        customerio-workspace-siteid: ${{ secrets[format('CUSTOMERIO_{0}_WORKSPACE_SITE_ID', matrix.apn-or-fcm)] }}
+        customerio-workspace-cdp-api-key: ${{ secrets[format('CUSTOMERIO_{0}_WORKSPACE_CDP_API_KEY', matrix.apn-or-fcm)] }}
         GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64: ${{ secrets.GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64 }}
         FIREBASE_APP_DISTRIBUTION_SERVICE_ACCOUNT_CREDS_B64: ${{ secrets.FIREBASE_APP_DISTRIBUTION_SERVICE_ACCOUNT_CREDS_B64 }}
-
-    # xcodebuild creates builds that include a timestamp in the name. In order for bloaty to read the build, we need to rename it to a static name.
-    - name: Rename the same app build to a static name that we can generate SDK size reports with
-      working-directory: Apps/${{ matrix.sample-app }}/build/
-      run: mv *.xcarchive App.xcarchive 
-
-    - name: Generate SDK size reports 
-      uses: ./.github/actions/generate-sdk-size-report      
-      if: ${{ matrix.sample-app == 'APN-UIKit' }}
-      id: generate-sdk-size-report
-      with:
-        apn-app-xcarchive-name: App.xcarchive
-        generate-main-diff-report: ${{ github.event_name == 'pull_request' }}
-
-    - name: Save the sample app build, if this is a main branch build 
-      if: github.ref == 'refs/heads/main'   
-      uses: ./.github/actions/main-sample-app-build 
-      with:
-        apn-app-xcarchive-name: App.xcarchive
 
     - name: Update sample builds PR comment with build information 
       if: ${{ github.event_name == 'pull_request' }}
@@ -164,8 +98,21 @@ jobs:
           * ${{ matrix.sample-app }}: Build failed. See [CI job logs](https://github.com/${{github.repository}}/actions/runs/${{github.run_id}}) to determine the issue and try re-building. 
         edit-mode: append # append new line to the existing PR comment to build a list of all sample app builds. 
 
+  generate-sdk-size-reports:
+    runs-on: macos-14
+    permissions:
+      pull-requests: write # to be able to comment on PR
+    name: Generate SDK size reports 
+    steps:
+    - uses: actions/checkout@v4
+
+    - uses: ./.github/actions/generate-sdk-size-report
+      id: generate-sdk-size-report
+      with:
+        GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64: ${{ secrets.GOOGLE_CLOUD_MATCH_READONLY_SERVICE_ACCOUNT_B64 }}
+
     - name: Find existing SDK size report comment, if there is one. 
-      if: github.event_name == 'pull_request' && matrix.sample-app == 'APN-UIKit'
+      if: github.event_name == 'pull_request'
       uses: peter-evans/find-comment@v3
       id: find-sdk-size-report-comment
       with:
@@ -174,7 +121,7 @@ jobs:
         body-includes: <!-- sdk size reports -->
 
     - name: Send SDK size reports to the PR for convenient viewing 
-      if: github.event_name == 'pull_request' && matrix.sample-app == 'APN-UIKit'
+      if: github.event_name == 'pull_request'
       uses: peter-evans/create-or-update-comment@v4
       with:
         comment-id: ${{ steps.find-sdk-size-report-comment.outputs.comment-id }}
@@ -200,4 +147,4 @@ jobs:
           ${{ steps.generate-sdk-size-report.outputs.sdk-size-diff-report }}
           ```
           
-          </details>
+          </details>

Apps/.gitignore

@@ -6,4 +6,5 @@ Pods/
 **/fastlane/report.xml
 **/fastlane/README.md
 BuildEnvironment.swift
-build/
+build/
+spm_packages/