Subscription oauth v2 #1033

federicocappelli · 2024-10-25T09:52:19Z

Task/Issue URL: https://app.asana.com/0/1205842942115003/1207991044706235/f
iOS PR: duckduckgo/iOS#3480
macOS PR: duckduckgo/macos-browser#3580
What kind of version bump will this require?: Major
CC: @miasma13

Description:

This PR introduces the use of OAuth V2 authentication in Privacy Pro Subscription.
The code changes are comprehensive due to the paradigm changes between the old access token lifecycle and the new JWT lifecycle.
The Subscription UI and UX should be unchanged.

Steps to test this PR:
Test all Privacy Pro Subscription features and UX, more details here

Internal references:

Software Engineering Expectations
Technical Design Template

…th_api_v2

# Conflicts: # Sources/Networking/README.md # Sources/Networking/v2/APIRequestErrorV2.swift # Sources/Networking/v2/APIRequestV2.swift # Sources/Networking/v2/APIResponseConstraints.swift # Sources/Networking/v2/APIService.swift # Sources/TestUtils/MockAPIService.swift # Tests/NetworkingTests/v2/APIServiceTests.swift

# Conflicts: # Sources/Subscription/Flows/AppStore/AppStoreAccountManagementFlow.swift

attempt to clean out tests output in CI reverted

federicocappelli · 2024-12-12T15:27:23Z

Sources/MaliciousSiteProtection/API/APIClient.swift

@@ -29,6 +29,7 @@ extension APIClient {
 extension APIClient: APIClient.Mockable {}

 public protocol APIClientEnvironment {
+    func queryItems(for requestType: APIRequestType) -> QueryItems


Adopting the improved Networking V2

federicocappelli · 2024-12-12T15:34:24Z

Sources/Networking/v2/Extensions/HTTPURLResponse+Cookie.swift

@@ -1,7 +1,7 @@
 //
-//  DecodableHelper.swift
+//  HTTPURLResponse+Cookie.swift


Note: github is mistaken here, not the same file at all

…com/duckduckgo/BrowserServicesKit into fcappelli/subscription_oauth_api_v2

miasma13 · 2024-12-13T14:05:50Z

Sources/Common/CodableHelper.swift

+    }
+}
+
+public typealias DecodableHelper = CodableHelper


Is this required?

Yes but just because the branch is long-living, 90% of the "dumb" merge conflicts I get are for this name change, I'll remove it after we merge this.

miasma13 · 2024-12-13T14:30:44Z

Sources/Subscription/Flows/Models/SubscriptionOptions.swift

+    /// The available features in the subscription based on the country and feature flags. Not based on user entitlements
+    let features: [SubscriptionOptions.Feature]
+
+    public init(platform: SubscriptionPlatformName, options: [SubscriptionOption], availableEntitlements: [SubscriptionEntitlement]) {


What is the reason for changing the naming here from features to availableEntitlements. This is confusing and against the naming conventions we agreed upon:

entitlements are permissions granted to user's account for accessing features (e.g. account has entitlement to use VPN)

features are services bundled together as a subscription plan, set of features defines the complete set of entitlements user is granted when subscribing

https://app.asana.com/0/1208524871249522/1208561749459707/f
https://app.asana.com/0/1208524871249522/1208561749459707/f

The reasons are the following:

The new source of truth for both user entitlements and subscription entitlements is a single function in SubscriptionManager, this function needs to return "something" that explains what features are available and if they are available for the specific user, this something natural name is Feature (entitlement + enabled or not), I tried to invent something different but the result was confusing.
How we obtain entitlements and features has changed a lot, getting stuck on the previous naming convention didn't make much sense.

The data we get from the backend is identical between user entitlement and subscription entitlement, they are all entitlement so I maintained the same approach, the moment we merge the two concept we get a SubscriptionFeature

The old Feature and Entitlement struct with var product was verbose for no reason, the only thingh we care of is the entitlement

The important concept here is that all of these entitlements and where they come from are internal implementation details, the only externally visible object is

public struct SubscriptionFeature: Equatable, CustomDebugStringConvertible { public var entitlement: SubscriptionEntitlement public var enabled: Bool ... }

miasma13 · 2024-12-13T14:57:14Z

Sources/Subscription/Managers/SubscriptionManager.swift

+    /// Returns the features available for the current subscription, a feature is enabled only if the user has the corresponding entitlement
+    /// - Parameter forceRefresh: ignore subscription and token cache and re-download everything
+    /// - Returns: An Array of SubscriptionFeature where each feature is enabled or disabled based on the user entitlements
+    public func currentSubscriptionFeatures(forceRefresh: Bool) async -> [SubscriptionFeature] {
+        guard isUserAuthenticated else { return [] }
+
+        if let subscriptionFeatureFlagger,
+           subscriptionFeatureFlagger.isFeatureOn(.isLaunchedROW) || subscriptionFeatureFlagger.isFeatureOn(.isLaunchedROWOverride) {
+            do {
+                let currentSubscription = try await getSubscription(cachePolicy: .returnCacheDataDontLoad)
+                let tokenContainer = try await getTokenContainer(policy: forceRefresh ? .localForceRefresh : .local)
+                let userEntitlements = tokenContainer.decodedAccessToken.subscriptionEntitlements
+                let availableFeatures = currentSubscription.features ?? [] // await subscriptionFeatureMappingCache.subscriptionFeatures(for: subscription.productId)
+
+                // Filter out the features that are not available because the user doesn't have the right entitlements
+                let result = availableFeatures.map({ featureEntitlement in
+                    let enabled = userEntitlements.contains(featureEntitlement)
+                    return SubscriptionFeature(entitlement: featureEntitlement, enabled: enabled)
+                })
+                Logger.subscription.log("""
+User entitlements: \(userEntitlements)
+Available Features: \(availableFeatures)
+Subscription features: \(result)
+""")
+                return result
+            } catch {
                return []
            }
        } else {
-            return [.networkProtection, .dataBrokerProtection, .identityTheftRestoration]
+            let result = [SubscriptionFeature(entitlement: .networkProtection, enabled: true),
+                          SubscriptionFeature(entitlement: .dataBrokerProtection, enabled: true),
+                          SubscriptionFeature(entitlement: .identityTheftRestoration, enabled: true)]
+            Logger.subscription.debug("Default Subscription features: \(result)")
+            return result
+        }
+    }


I do not fully agree with this method. Subscription feature as a concept cannot be enabled/disabled - it is either present or not. A subscription is defined as being a bundle of features, e.g. "US Privacy Pro" has VPN, PIR, ITR, "ROW Privacy PRO" has VPN, ITR-ROW.

So where this implementation diverges from the old one in SubscriptionFeatureMappingCache it should not filter it through entitlements. It is the set of features as defined per subscription ID and fetched from BE API.

Also what is worth, this method is to be used by client apps to determine which parts of the UI are to be shown for the currently active subscription.

| A subscription is defined as being a bundle of features
yes, in the backend. We enhance the "subscription features" by adding additional info (available or not to the user)

I'm not sure where's the problem here, everything you described is right and it's how is currently implemented, in fact, the number/type of features returned by this function is dictated by the entitlement (backend name) present in the subscription and they are marked enabled/disabled by the user capabilities.

As you say this function user is the client's UI that needs a way to decide which features displaying (here is the name) and if they are disabled or not :) Exactly what this function provides.

Note that SubscriptionManager (in this function in particular) is the only entity that has the right to manipulate entitlements and create Features, nobody else touches them so the joining of user entitlements and subscription entitlements needs to happen here.

I think the misunderstanding here is that you are not separating the internal implementation from the Subscription framework interface. The models' name and functions' contract should be dictated by the need of the user (in this case the App's UI) not from some internal implementation ideas.

miasma13 · 2024-12-13T15:05:02Z

Sources/Subscription/Managers/SubscriptionManager.swift

+    public func isFeatureActive(_ entitlement: SubscriptionEntitlement) async -> Bool {
+        let currentFeatures = await currentSubscriptionFeatures(forceRefresh: false)
+        return currentFeatures.contains { feature in
+            feature.entitlement == entitlement && feature.enabled


Similar as comment above. I strongly would encourage to stick to naming conventions already established and rename it to hasEntitlement(_:). Also there should be no logic crosschecking it with the entitlements here, just simple check for entitlements array if one is present and return true or false.
I think I see what you tried to achieve here but the whole quite simple concept becomes convoluted without reason. Features in the subscription define what UI-wise is part of current subscription, and Entitlements define which parts user currently has access to within the scope of current subscription feature set.

As explained above hasEntitlement(_:) doesn't make sense anymore, nobody outside SubscriptionManager can or should know/care about internal implementation details (The S of SOLID principles) so it's the duty of currentSubscriptionFeatures to provide the right information needed from the UI, we can't ask the users (main apps) to check if the use entitlement is present, get the subscription entitlements, compare them and then update the UI.

THISISDINOSAUR · 2024-12-13T16:06:37Z

There's basically no PIR stuff in BSK at the moment

…cases

diegoreymendez

Adding some comments. Still going through the code.

Sources/NetworkProtection/Diagnostics/NetworkProtectionConnectionTester.swift

Sources/Common/KeychainType.swift

diegoreymendez · 2024-12-20T16:07:22Z

Sources/Common/UserDefaultsCache.swift

-    let logger = { Logger(subsystem: Bundle.main.bundleIdentifier ?? "DuckDuckGo", category: "UserDefaultsCache") }()
+    let logger = { Logger(subsystem: "UserDefaultsCache", category: "") }()


This will remove these entries from VPN log export. Is there a need to make this change?

I didn't know the exported was actively filtering for this, the reason for changing it is that, As for Apple's documentation: The subsystem string identifies a large functional area within your app or apps. this can be interpreted in several ways but all other Loggers' subsystems we declare are stuff like "Subscription", "Content Blocking" etc.
Additionally, logs already have the app bundle identifier/app name, so this field was a duplication.
Is it easy to change the log export? if not I can revert this

...lerErrorMessageObserver/ControllerErrorMesssageObserverThroughDistributedNotifications.swift

Sources/NetworkProtection/PacketTunnelProvider.swift

diegoreymendez · 2024-12-20T16:45:05Z

Sources/NetworkProtection/PacketTunnelProvider.swift

-            try load(options: startupOptions)
-
-            if (try? tokenStore.fetchToken()) == nil {
-                throw TunnelError.startingTunnelWithoutAuthToken
-            }
+            try await load(options: startupOptions)
+            Logger.networkProtection.log("Startup options loaded correctly")


The error handling needs to be improved here. Previously we could only have startingTunnelWithoutAuthToken, but now we can get a number of different possible errors from tokenProvider.adopt and tokenProvider.getTokenContainer.

These errors should ideally implement CustomNSError and include underlying error information. See PacketTunnelProvider.TunnelError for reference.

The reason this is important is because it's what'll allow us to debug tunnel start issues coming from these changes, and understand what's going on.

Understood, atm getTokenContainer can fail for a variety of reasons, from storage to API, decoding and decryption errors, we don't have a single set Error so implementing this is quite difficult, should I wrap any possible error into a higher level TunnelError?

federicocappelli added 30 commits September 6, 2024 14:44

Networking V2 implemented and tested

52c4e2f

networking v2 refinements and unit tests

6deb71d

headers renamed

aecc91b

mocks, lint

23800c2

api configuration removed and data moved to api request, lint

d029027

query items changed and tests fixed

ef4949c

added unit test

0dbc46e

Merge branch 'main' into fcappelli/networkingV2

00bc2b2

privacy ref repo updated

0000972

pkg updated

309770b

comments

d502223

Documentation + comments and improvements

0ea4332

Doc

3101949

Merge branch 'main' into fcappelli/networkingV2

b20199d

v1 restore

a11b58b

v1 restore

46a7423

tests fix

a7e2cae

comment fix

99f80f7

auth client, service etc

d3ca645

authorise call + real tests

193792f

authorisation API done

969514d

error handling fixed

bb8fb82

sendotp call

6ea42ad

renaming

237c90e

Merge branch 'fcappelli/networkingV2' into fcappelli/subscription_oau…

ea3d225

…th_api_v2

login api call

48091f9

more APIs

8c70535

access token api requests

0b246d0

optimisations

aa00362

federicocappelli added 7 commits December 12, 2024 10:44

backward compatibility typealias

975eb88

small cleanups

2cd4bab

Merge branch 'main' into fcappelli/subscription_oauth_api_v2

43774ad

# Conflicts: # Sources/Subscription/Flows/AppStore/AppStoreAccountManagementFlow.swift

extension moved from macos to bsk

30e74c8

code comments + sub restore improvements

57fc80b

mock fixed

50d4d64

Update pr.yml

b586366

attempt to clean out tests output in CI reverted

federicocappelli commented Dec 12, 2024

View reviewed changes

federicocappelli added 4 commits December 12, 2024 15:41

code cleanup

1292c9d

Merge branch 'fcappelli/subscription_oauth_api_v2' of https://github.…

f12498a

…com/duckduckgo/BrowserServicesKit into fcappelli/subscription_oauth_api_v2

stripe success integration test

a942eea

code cleanup

3ce186f

federicocappelli marked this pull request as ready for review December 12, 2024 17:42

federicocappelli requested review from miasma13, samsymons, diegoreymendez and THISISDINOSAUR December 13, 2024 10:52

date extension improved and unit tests added

77e4a93

miasma13 reviewed Dec 13, 2024

View reviewed changes

lint

8fec94f

miasma13 reviewed Dec 13, 2024

View reviewed changes

federicocappelli added 2 commits December 16, 2024 14:37

DBP cleanup

2f82911

Subscription purchase integration tests for apple and stripe + error …

8314ec0

…cases

diegoreymendez reviewed Dec 20, 2024

View reviewed changes

federicocappelli added 2 commits December 23, 2024 13:20

code cleanup and log fixes + get token now uses localValid everywhere

771d444

Adopt function improvements

dfc9582

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Subscription oauth v2 #1033

Subscription oauth v2 #1033

federicocappelli commented Oct 25, 2024 •

edited

Loading

federicocappelli Dec 12, 2024

federicocappelli Dec 12, 2024

miasma13 Dec 13, 2024

federicocappelli Dec 13, 2024

miasma13 Dec 13, 2024

federicocappelli Dec 13, 2024

miasma13 Dec 13, 2024

miasma13 Dec 13, 2024

federicocappelli Dec 13, 2024

miasma13 Dec 13, 2024

federicocappelli Dec 13, 2024

THISISDINOSAUR commented Dec 13, 2024

diegoreymendez left a comment

diegoreymendez Dec 20, 2024

federicocappelli Dec 23, 2024

diegoreymendez Dec 20, 2024

federicocappelli Dec 23, 2024

		let logger = { Logger(subsystem: Bundle.main.bundleIdentifier ?? "DuckDuckGo", category: "UserDefaultsCache") }()
		let logger = { Logger(subsystem: "UserDefaultsCache", category: "") }()

Subscription oauth v2 #1033

Are you sure you want to change the base?

Subscription oauth v2 #1033

Conversation

federicocappelli commented Oct 25, 2024 • edited Loading

Internal references:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

THISISDINOSAUR commented Dec 13, 2024

diegoreymendez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

federicocappelli commented Oct 25, 2024 •

edited

Loading