-
Notifications
You must be signed in to change notification settings - Fork 271
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Apply BSA diffs to the database #2229
base: master
Are you sure you want to change the base?
Conversation
import org.junit.jupiter.api.Test; | ||
|
||
/** Unit tests for {@link NonBlockedDomain}. */ | ||
class NonBlockedDomainTest { |
Check notice
Code scanning / CodeQL
Unused classes and interfaces Note test
ImmutableMap<BlockList, String> fetchedChecksums = | ||
ImmutableMap.of(BLOCK, block.peekChecksum(), BLOCK_PLUS, blockPlus.peekChecksum()); |
Check notice
Code scanning / CodeQL
Unread local variable Note
ImmutableMap<BlockList, String> prevChecksums = | ||
schedule | ||
.latestCompleted() | ||
.map(DownloadSchedule.CompletedJob::checksums) | ||
.orElseGet(ImmutableMap::of); |
Check notice
Code scanning / CodeQL
Unread local variable Note
checkArgument(columns.size() == 2, "Invalid line: [%s]", line); | ||
checkArgument(!Strings.isNullOrEmpty(columns.get(0)), "Missing label in line: [%s]", line); | ||
ImmutableList<Long> orderIds = | ||
ORDER_SPLITTER.splitToStream(columns.get(1)).map(Long::valueOf).collect(toImmutableList()); |
Check notice
Code scanning / CodeQL
Missing catch of NumberFormatException Note
import org.junit.jupiter.api.Test; | ||
|
||
/** Unit tests for {@link Label}. */ | ||
class LabelTest { |
Check notice
Code scanning / CodeQL
Unused classes and interfaces Note test
} | ||
|
||
static ImmutableList<NonBlockedDomain> tallyNonBlockedDomainsForNewLabels( | ||
ImmutableList<Label> labels, IdnChecker idnChecker, DownloadSchedule schedule, DateTime now) { |
Check notice
Code scanning / CodeQL
Useless parameter Note
9be8b41
to
a76171a
Compare
import org.testcontainers.shaded.com.google.common.collect.ImmutableList; | ||
|
||
/** Unit tests for {@link JsonSerializations}. */ | ||
class JsonSerializationsTest { |
Check notice
Code scanning / CodeQL
Unused classes and interfaces Note test
core/src/main/java/google/registry/bsa/api/BsaReportSender.java
Dismissed
Show dismissed
Hide dismissed
42eb420
to
161b7d1
Compare
161b7d1
to
f9e652f
Compare
Add the BsaDomainRefresh class which tracks the refresh actions. The refresh actions checks for changes in the set of registered and reserved domains, which are called unblockables to BSA.
Compare the latest BSA download and the previous one to discover: * New and deleted orders * New and deleted labels as well as konwn labels referenced by new orders The diff is stored on GCS as two files, one for orders and one for diffs.
Add a step in BsaDownloadAction to apply the diffs from the most recent download. This includes: - Add new labels and remove deleted labels to the `BsaLabel` table. - Add new entries to the `BsaDomainInUse` table if applicable. These are registered/reserved domains that match the new labels. - Generate list of domains that are not blocked and save them in a GCS file. These include all domains from the previous step as well as domains derived from invalid labels. More tests are needed but this PR successfully processed a real download.
f9e652f
to
e2eb621
Compare
Add a step in BsaDownloadAction to apply the diffs from the most recent
download. This includes:
BsaLabel
table.BsaDomainInUse
table if applicable. These areregistered/reserved domains that match the new labels.
file. These include all domains from the previous step as well as
domains derived from invalid labels.
More tests are needed but this PR successfully processed a real
download.
This change is