Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update ClickToLoad to use shared C-S-S #2162

Merged
merged 62 commits into from
Jun 14, 2024
Merged
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
01c8a53
swap embedded user script with CSS one
ladamski Feb 6, 2024
cf561f1
fire "displayCLickToLoadPlaceholders" on load completion
ladamski Feb 7, 2024
98cb9c5
disable old CTL user script
ladamski Feb 12, 2024
8a3f6d6
test - trigger CTL on FB tracker match
ladamski Feb 12, 2024
367f031
switch to isolated scope, hook up overlay trigger
ladamski Feb 13, 2024
e19f103
use v6 TDS, updated config
ladamski Mar 2, 2024
ae481dc
move CTL code to subfolder
ladamski Mar 2, 2024
50f1411
update CTL rule management to use main TDS
ladamski Mar 2, 2024
b603ed7
WIP rule splitting for custom actions
ladamski Mar 5, 2024
f971774
cleanup FB protection state management
ladamski Mar 7, 2024
daf57ad
update embedded config and TDS
ladamski Mar 16, 2024
0d2c7a4
remove dead code
ladamski Mar 16, 2024
3a0c7fd
switch back to separate blocking CTL rule set
ladamski Mar 18, 2024
7156d78
switch back to previous CTL rules, part 2
ladamski Mar 20, 2024
fd7287e
combine rules for surrogates user script
ladamski Mar 21, 2024
7025ba1
cleanup surrogate code to remove need for vars
ladamski Apr 5, 2024
d3d0c87
update embeds
ladamski Apr 11, 2024
4879680
update config clickToPlay -> clickToLoad
ladamski Apr 11, 2024
909e53b
burn with fire (dead code)
ladamski Apr 11, 2024
be1a4ea
dead code cleanup part 2
ladamski Apr 15, 2024
12cce92
clarify method naming
ladamski Apr 15, 2024
7ed7200
handle empty ruleset in in surrogate generation (unit tests)
ladamski Apr 16, 2024
b249af6
use fallback TDS for CTL test vs old embedded CTL TDS
ladamski Apr 17, 2024
a5e1287
bump TDS test to v6
ladamski Apr 17, 2024
a93d49d
remove remapping of custom action .blockCtlFB to .block
ladamski Apr 19, 2024
5069df9
remove dead fonts, post-merge cleanup
ladamski Apr 23, 2024
f77d0c7
TEMP disable CTL assertions while testing
ladamski Apr 24, 2024
7c3f672
address @MainActor warnings, remove debug spam
ladamski Apr 24, 2024
1bc59ed
lint nit
ladamski Apr 24, 2024
841de53
try special casing NavigationType.backForward for CTL
ladamski Apr 24, 2024
838d6f8
remove MD5 func
ladamski Apr 30, 2024
abdb4f0
move FB entity string to const in FBProtectionTabExtension
ladamski Apr 30, 2024
1999c4a
update splitter logic per PR review, fix test
ladamski May 1, 2024
3ca418b
Update CTL rule splitting logic with alternate path (#2733)
ladamski May 1, 2024
95a798c
update rule combining logic per last rule splitting update
ladamski May 2, 2024
1856498
remove dead ClickToLoadModel code
ladamski May 21, 2024
4a05050
Update ClickToLoadRulesSplitter.swift
ladamski May 21, 2024
0072d09
code review nits
ladamski May 21, 2024
2a61134
Merge branch 'main' into la/update-ctl
ladamski May 21, 2024
d19a19f
cleanup FIXME
ladamski May 21, 2024
223b434
update to BSK WIP branch
ladamski May 21, 2024
cb42dd6
Update proj file and BSK ref
jaceklyp May 21, 2024
098696e
remove unused var
ladamski May 22, 2024
b01f13e
update CTL action to blockCTLFB
ladamski May 23, 2024
084d24d
add CTL test for rules splitting
ladamski May 23, 2024
2a2cf71
test nits
ladamski May 23, 2024
85cc309
move ClickToLoadRulesSpliter to BSK
ladamski May 31, 2024
3673f70
Update ClickToLoadTDSTests.swift
ladamski May 31, 2024
6ca5394
move ClickToLoadRulesSplitter to BSK
ladamski May 31, 2024
d55102c
review nits
ladamski May 31, 2024
77993dd
Update ClickToLoadTDSTests.swift
ladamski Jun 3, 2024
13af034
test tweaks
ladamski Jun 7, 2024
f5c56fd
Update ClickToLoadTDSTests.swift
ladamski Jun 8, 2024
24566fd
remove forward/back nav exception in FB protections
ladamski Jun 12, 2024
98c095b
revert config URL to prod and update embeds
ladamski Jun 13, 2024
2d04a7c
release BSK
ladamski Jun 14, 2024
0479655
update BSK dep
ladamski Jun 14, 2024
84d76eb
Merge branch 'main' into la/update-ctl
ladamski Jun 14, 2024
ef74977
Restore NetP Package.swift
bwaresiak Jun 14, 2024
388d73e
version bump to 1.93.0
ladamski Jun 14, 2024
a0ab778
Revert "version bump to 1.93.0"
ladamski Jun 14, 2024
5e5feb1
nit
ladamski Jun 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update CTL rule management to use main TDS
  • Loading branch information
ladamski committed Apr 23, 2024
commit 50f14111a9302d9cd9583e65bf60d16036ab7130
122 changes: 122 additions & 0 deletions DuckDuckGo/ContentBlocker/ClickToLoad/ClickToLoadRulesSplitter.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
//
// ClickToLoadRulesSplitter.swift
// DuckDuckGo
//
// Copyright © 2023 DuckDuckGo. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

import TrackerRadarKit
import BrowserServicesKit

struct ClickToLoadRulesSplitter {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any tests for the splitter, I'd expect it to be unit tested really

This comment was marked as resolved.


public enum Constant {

public static let clickToLoadRuleListPrefix = "CTL_"

}

private let rulesList: ContentBlockerRulesList

init(rulesList: ContentBlockerRulesList) {
self.rulesList = rulesList
}

func split() -> (withoutBlockCTL: ContentBlockerRulesList, withBlockCTL: ContentBlockerRulesList)? {
let splitTDS = rulesList.trackerData != nil ? split(trackerData: rulesList.trackerData!) : nil
let splitFallbackTDS = split(trackerData: rulesList.fallbackTrackerData)

if splitTDS != nil || splitFallbackTDS != nil {
return (
ContentBlockerRulesList(name: rulesList.name,
trackerData: splitTDS?.withoutBlockCTL ?? rulesList.trackerData,
fallbackTrackerData: splitFallbackTDS?.withoutBlockCTL ?? rulesList.fallbackTrackerData),
ladamski marked this conversation as resolved.
Show resolved Hide resolved
ContentBlockerRulesList(name: "XD",
trackerData: splitTDS?.withBlockCTL ?? rulesList.trackerData,
fallbackTrackerData: splitFallbackTDS?.withBlockCTL ?? rulesList.fallbackTrackerData)
)
}
return nil
}

private func split(trackerData: TrackerDataManager.DataSet) -> (withoutBlockCTL: TrackerDataManager.DataSet, withBlockCTL: TrackerDataManager.DataSet)? {
let trackersWithBlockCTL = filterTrackersByBlockCTLAction(trackerData.tds.trackers, hasBlockCTL: true)

if !trackersWithBlockCTL.isEmpty {
let trackersWithoutBlockCTL = filterTrackersByBlockCTLAction(trackerData.tds.trackers, hasBlockCTL: false)
let trackerDataWithoutBlockCTL = makeTrackerData(using: trackersWithoutBlockCTL, originalTDS: trackerData.tds)
let trackerDataWithBlockCTL = makeTrackerData(using: trackersWithBlockCTL, originalTDS: trackerData.tds)

return (
(tds: trackerDataWithoutBlockCTL, etag: Constant.clickToLoadRuleListPrefix + trackerData.etag),
(tds: trackerDataWithBlockCTL, etag: Constant.clickToLoadRuleListPrefix + trackerData.etag)
)
}
return nil
}

private func makeTrackerData(using trackers: [String: KnownTracker], originalTDS: TrackerData) -> TrackerData {
let entities = originalTDS.extractEntities(for: trackers)
let domains = extractDomains(from: entities)
return TrackerData(trackers: trackers,
entities: entities,
domains: domains,
cnames: originalTDS.cnames)
}

private func filterTrackersByBlockCTLAction(_ trackers: [String: KnownTracker], hasBlockCTL: Bool) -> [String: KnownTracker] {
trackers.filter { (_, tracker) in tracker.containsCTLActions == hasBlockCTL }
}

private func extractDomains(from entities: [String: Entity]) -> [String: String] {
var domains = [String: String]()
for entity in entities {
for domain in entity.value.domains ?? [] {
domains[domain] = entity.key
}
}
ladamski marked this conversation as resolved.
Show resolved Hide resolved
return domains
}

}

private extension TrackerData {

func extractEntities(for trackers: [String: KnownTracker]) -> [String: Entity] {
let trackerOwners = Set(trackers.values.compactMap { $0.owner?.name })
let entities = entities.filter { trackerOwners.contains($0.key) }
return entities
}

}

private extension KnownTracker {

var containsCTLActions: Bool {
if let defaultAction = defaultAction, defaultAction == .blockCtlFB {
return true
}

if let rules = rules {
for rule in rules {
if let action = rule.action, action == .blockCtlFB {
return true
}
}
}
return false
}

}
21 changes: 12 additions & 9 deletions DuckDuckGo/ContentBlocker/ContentBlockerRulesLists.swift
Original file line number Diff line number Diff line change
@@ -66,21 +66,24 @@ final class ContentBlockerRulesLists: DefaultContentBlockerRulesListsSource {
let tdsRulesIndex = result.firstIndex(where: { $0.name == DefaultContentBlockerRulesListsSource.Constants.trackerDataSetRulesListName }) {
let tdsRules = result[tdsRulesIndex]
let allowlistedTrackerNames = adClickAttribution.allowlist.map { $0.entity }
let splitter = AdClickAttributionRulesSplitter(rulesList: tdsRules, allowlistedTrackerNames: allowlistedTrackerNames)
if let splitRules = splitter.split() {
let adSplitter = AdClickAttributionRulesSplitter(rulesList: tdsRules, allowlistedTrackerNames: allowlistedTrackerNames)
if let splitRules = adSplitter.split() {
result.remove(at: tdsRulesIndex)
result.append(splitRules.0)
result.append(splitRules.1)
}
}

// Add new ones
let etag = MD5(data: Self.fbTrackerDataFile)
ladamski marked this conversation as resolved.
Show resolved Hide resolved
let dataSet: TrackerDataManager.DataSet = TrackerDataManager.DataSet(Self.fbTrackerDataSet, etag)
let CTLRulesList = ContentBlockerRulesList(name: Constants.clickToLoadRulesListName,
trackerData: nil,
fallbackTrackerData: dataSet)
result.append(CTLRulesList)
// split CTL rules so they can be managed separately from the main list when user clicks through a CTL dialog
if let tdsRulesIndex = result.firstIndex(where: { $0.name == DefaultContentBlockerRulesListsSource.Constants.trackerDataSetRulesListName }) {
let tdsRules = result[tdsRulesIndex]
let ctlSplitter = ClickToLoadRulesSplitter(rulesList: tdsRules)
if let splitRules = ctlSplitter.split() {
result.remove(at: tdsRulesIndex)
result.append(splitRules.withoutBlockCTL)
result.append(splitRules.withBlockCTL)
}
}

return result
}