Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

return user measurement #2011

Merged
merged 7 commits into from
Sep 15, 2023
Merged

return user measurement #2011

merged 7 commits into from
Sep 15, 2023

Conversation

brindy
Copy link
Contributor

@brindy brindy commented Sep 14, 2023

Task/Issue URL: https://app.asana.com/0/72649045549333/1204818349335868/f
Tech Design URL:
CC:

Description:

Allows us to measure the impact of returning users on retention.

Steps to test this PR:

Happy flow:

  1. Install and run the app from develop at least once.
  2. Delete the app (but do not reset the simulator) and then checkout this branch.
  3. Add a breakpoint to StatisticsLoader on requestExti function (or you can use a proxy to see the call to the /exit endpoint)
  4. Run the app and step over. The installAtb should be today's ATB plus the ru variant, e.g. v395-2ru
  5. Let the app run as normal (but keep the breakpoint for now)
  6. In the console filter on pixel.
  7. Delete the app again, then run it.
  8. When it hits the break point let it run.
  9. You should see a line in the console like: Pixel fired m_return_user ["old_atb": "v395-2", "new_atb": "v395-2"] (variant is not included)
  10. Reset your simulator using xcrun simctl erase all and run the app again.
  11. This time there should be no ru variant on the call to /exti and there should be no m_return_user pixel.

Update ATB (ie an old install gets generalised):

  1. Use a proxy that lets you breakpoint on calls (e.g. ProxyMan)
  2. Create a https breakpoint for duckduckgo.com/atb.js
  3. Add a breakpoint in the code on ReturnUserMeasurement.updateStoredATB
  4. Run the app and run all breakpoints.
  5. Perform a search then in the https breakpoint for atb.js return the following json: { "majorVersion": 395, "for_more_info": "https://duck.co/help/privacy/atb", "version": "v395-3", "minorVersion": 3, "updateVersion": "v20-1" }

Return User Pixel accuracy:

  1. Use a proxy that lets you breakpoint on calls (e.g. ProxyMan)
  2. Create a https breakpoint for duckduckgo.com/atb.js
  3. In the https breakpoint for atb.js return the following json: { "majorVersion": 400, "for_more_info": "https://duck.co/help/privacy/atb", "version": "v400-1", "minorVersion": 1 }
  4. Run the app.
  5. You should see a line in the console like: Pixel fired m_return_user ["old_atb": "v395-2", "new_atb": "v400-1"] (variant is not included and new atb is from the JSON supplied in the https breakpoint)

Debug pixels:

  1. Remove breakpoints from above
  2. Delete the app.
  3. Update the query in ReturnUserMeasurement.writeSecureATB and add a nonsense attribute, e.g. "none": "sense".
  4. Run the app. You should see a m_debug_return_user_add_atb pixel with an error code parameter.
  5. Delete the app.
  6. Update the second query in ReturnUserMeasurement.writeSecureATB and add a nonsense attribute, e.g. "none": "sense".
  7. Run the app. You should see a m_debug_return_user_update_atb pixel with an error code parameter.
  8. Delete the app.
  9. Update the query in ReturnUserMeasurement.readSecureATB and replace kSecMatchLimitOne with "rubbish".
  10. Run the app. You should see a m_debug_return_user_read_atb pixel with an error code parameter.

Device Testing:

  • iPhone
  • iPad

OS Testing:

  • iOS 14
  • iOS 15
  • iOS 16

Internal references:

Software Engineering Expectations
Technical Design Template

Copy link
Contributor

@diegoreymendez diegoreymendez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great to me. Nice work!

@brindy brindy merged commit 0e95736 into develop Sep 15, 2023
8 checks passed
@brindy brindy deleted the brindy/return-user-measurement branch September 15, 2023 16:06
@brindy brindy mentioned this pull request Nov 10, 2023
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants