Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Login Assistance: Update the "migrate CTA when login fails" experiment #92245

Merged
merged 2 commits into from
Jul 4, 2024

Conversation

m1r0
Copy link
Member

@m1r0 m1r0 commented Jul 1, 2024

Resolves #92246

Proposed Changes

  • Use the recognized_logins cookie to detect existing users and don't show them the CTA.
  • Make sure the CTA is displayed only when the form responds with unknown_user. This fixes a bug where the CTA is displayed when an empty form is submitted and the calypso_login_block_login_form_get_auth_type_failure (error_code: unknown_user) experiment exposure event was fired previously.
  • Update the experiment name.

image

Why are these changes being made?

  • We want to re-run the previous experiment with a few updates. More context can be found here: pdxWSz-1yz-p2#comment-488

Testing Instructions

  • In an incognito window, go to http://calypso.localhost:3000/log-in and enter a non-existing email.
  • Verify that a "User does not exist." error is displayed.
  • Close and re-open the page.
  • Now manually assign yourself to the "Treatment" group of the 21864-explat-experiment following this guide: PCYsg-SwK-p2 (Manually assign with Abacus assignment bookmarklet). Check the video below to see how it works. Note that you may have to restart the incognito session if the bookmarklet doesn't work.
  • After assigning yourself to the "Treatment" group, enter a non-existing email again.
  • Verify that you see the "User does not exist." error and the migration notice.
  • Verify that the migration notice CTA points to https://wordpress.com/move/.
  • Manually add a recognized_logins cookie with some value to the calypso.localhost domain.
  • Refresh the page, enter a non-existing email again, and make sure the CTA is not displayed.
  • If you want to do a proper test for the recognized_logins cookie, setup your local Calypso to run on wordpress.com (where the cookie is set by the public api) following this guide: PCYsg-5YE-p2. Or change the domain where the cookie is set on your sandbox.
rfdJOj.mp4

Pre-merge Checklist

  • Has the general commit checklist been followed? (PCYsg-hS-p2)
  • Have you written new tests for your changes?
  • Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
  • Have you checked for TypeScript, React or other console errors?
  • Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
  • Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
  • For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

@matticbot
Copy link
Contributor

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

App Entrypoints (~43 bytes added 📈 [gzipped])

name         parsed_size           gzip_size
entry-login       +119 B  (+0.0%)      +43 B  (+0.0%)

Common code that is always downloaded and parsed every time the app is loaded, no matter which route is used.

Sections (~78 bytes added 📈 [gzipped])

name             parsed_size           gzip_size
jetpack-connect       +119 B  (+0.0%)      +46 B  (+0.0%)
security               +78 B  (+0.0%)      +32 B  (+0.0%)

Sections contain code specific for a given set of routes. Is downloaded and parsed only when a particular route is navigated to.

Async-loaded Components (~79 bytes added 📈 [gzipped])

name                          parsed_size           gzip_size
async-load-design-blocks           +119 B  (+0.0%)      +46 B  (+0.0%)
async-load-signup-steps-user        +78 B  (+0.0%)      +33 B  (+0.1%)

React components that are loaded lazily, when a certain part of UI is displayed for the first time.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

@m1r0 m1r0 self-assigned this Jul 2, 2024
@matticbot
Copy link
Contributor

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

  • notifications
  • wpcom-block-editor

To test WordPress.com changes, run install-plugin.sh $pluginSlug update/failed-login-migrate-cta-experiment on your sandbox.

@m1r0 m1r0 marked this pull request as ready for review July 2, 2024 15:25
@m1r0 m1r0 requested a review from a team July 2, 2024 15:25
@matticbot matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Jul 2, 2024
Copy link
Contributor

@donnapep donnapep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, though I didn't know how to test the unknown_user case at all. I presume that you tested that bit?

@m1r0
Copy link
Member Author

m1r0 commented Jul 4, 2024

Thanks, Donna!

I didn't know how to test the unknown_user case at all. I presume that you tested that bit?

Yep, that one should be fine if you saw the notice while testing with an unknown user. 👍

@m1r0 m1r0 merged commit e713d16 into trunk Jul 4, 2024
30 of 34 checks passed
@m1r0 m1r0 deleted the update/failed-login-migrate-cta-experiment branch July 4, 2024 08:23
@github-actions github-actions bot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Jul 4, 2024
m1r0 added a commit that referenced this pull request Jul 31, 2024
m1r0 added a commit that referenced this pull request Aug 7, 2024
…riment (#93146)

* Revert "Login Assistance: Update the "migrate CTA when login fails" experiment (#92245)"

This reverts commit e713d16.

* Revert "Login: Fix migrate notice events not triggering (#91607)"

This reverts commit 9daba74.

* Revert "Login Assistance: Show migrate CTA when login fails (#91188)"

This reverts commit 0f754b1.

* Keep the `getCookies` function
gabrielcaires pushed a commit that referenced this pull request Aug 8, 2024
…riment (#93146)

* Revert "Login Assistance: Update the "migrate CTA when login fails" experiment (#92245)"

This reverts commit e713d16.

* Revert "Login: Fix migrate notice events not triggering (#91607)"

This reverts commit 9daba74.

* Revert "Login Assistance: Show migrate CTA when login fails (#91188)"

This reverts commit 0f754b1.

* Keep the `getCookies` function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Login Assistance: Don't show the migrate CTA to existing users
3 participants