[Spark] Replaces startTransaction calls with catalogTable overload #2125
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
very nice ! |
lzlfred
approved these changes
Sep 29, 2023
5 tasks
vkorukanti
pushed a commit
to vkorukanti/delta
that referenced
this pull request
Oct 3, 2023
This PR replaces all calls to OptimisticTransaction.startTransaction() with calls to OptimisticTransaction.startTransaction(Option[CatalogTable]). This PR is part of delta-io#2105 and ensures that transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog. This is a small refactoring change so existing test coverage is sufficient. No Closes delta-io#2125 GitOrigin-RevId: 014a459275e5fec5bfc51ff143563b2949c607c3
vkorukanti
pushed a commit
to vkorukanti/delta
that referenced
this pull request
Oct 3, 2023
This PR replaces all calls to OptimisticTransaction.startTransaction() with calls to OptimisticTransaction.startTransaction(Option[CatalogTable]). This PR is part of delta-io#2105 and ensures that transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog. This is a small refactoring change so existing test coverage is sufficient. No Closes delta-io#2125 GitOrigin-RevId: 014a459275e5fec5bfc51ff143563b2949c607c3
Kimahriman
pushed a commit
to Kimahriman/delta
that referenced
this pull request
Oct 3, 2023
This PR replaces all calls to OptimisticTransaction.startTransaction() with calls to OptimisticTransaction.startTransaction(Option[CatalogTable]). This PR is part of delta-io#2105 and ensures that transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog. This is a small refactoring change so existing test coverage is sufficient. No Closes delta-io#2125 GitOrigin-RevId: 014a459275e5fec5bfc51ff143563b2949c607c3
vkorukanti
pushed a commit
to vkorukanti/delta
that referenced
this pull request
Oct 6, 2023
…verload This PR replaces all calls to DeltaLog.startTransaction(Snapshot) with calls to DeltaLog.startTransaction(Option[CatalogTable], Snapshot). This PR is part of delta-io#2105 and a follow-up to delta-io#2125. It makes sure that transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog. This is a small refactoring change so existing test coverage is sufficient. No Closes delta-io#2126 GitOrigin-RevId: d82787c64979a2dd4a363bf92a1640b7635ec02e
scottsand-db
pushed a commit
that referenced
this pull request
Oct 6, 2023
…verload This PR replaces all calls to DeltaLog.startTransaction(Snapshot) with calls to DeltaLog.startTransaction(Option[CatalogTable], Snapshot). This PR is part of #2105 and a follow-up to #2125. It makes sure that transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog. This is a small refactoring change so existing test coverage is sufficient. No Closes #2126 GitOrigin-RevId: d82787c64979a2dd4a363bf92a1640b7635ec02e
vkorukanti
pushed a commit
to vkorukanti/delta
that referenced
this pull request
Oct 26, 2023
…ith catalogTable overload #### Which Delta project/connector is this regarding? -Spark - [ ] Standalone - [ ] Flink - [ ] Kernel - [ ] Other (fill in here) ## Description This PR replaces the call to DeltaLog.startTransaction() in StatisticsCollection.recompute with calls to DeltaLog.startTransaction(Option[CatalogTable], Snapshot). This PR is part of delta-io#2105 and a follow-up to delta-io#2125, to ensure that all transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog. This PR also introduces a new helper in DeltaTestImplicits, which allows unit test call sites to still call the old version of StatisticsCollection.recompute and passes None as the catalogTable. This implicit should only be used if the test really only runs against a path-based Delta table and so no catalogTable is present. This is a small refactoring change so existing test coverage is sufficient. ## Does this PR introduce _any_ user-facing changes? No Closes delta-io#2217 GitOrigin-RevId: bc3783d298de7d7ad442ac347042d8fc7820afbe
xupefei
pushed a commit
to xupefei/delta
that referenced
this pull request
Oct 31, 2023
…verload This PR replaces all calls to DeltaLog.startTransaction(Snapshot) with calls to DeltaLog.startTransaction(Option[CatalogTable], Snapshot). This PR is part of delta-io#2105 and a follow-up to delta-io#2125. It makes sure that transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog. This is a small refactoring change so existing test coverage is sufficient. No Closes delta-io#2126 GitOrigin-RevId: d82787c64979a2dd4a363bf92a1640b7635ec02e
xupefei
pushed a commit
to xupefei/delta
that referenced
this pull request
Oct 31, 2023
…ith catalogTable overload #### Which Delta project/connector is this regarding? -Spark - [ ] Standalone - [ ] Flink - [ ] Kernel - [ ] Other (fill in here) ## Description This PR replaces the call to DeltaLog.startTransaction() in StatisticsCollection.recompute with calls to DeltaLog.startTransaction(Option[CatalogTable], Snapshot). This PR is part of delta-io#2105 and a follow-up to delta-io#2125, to ensure that all transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog. This PR also introduces a new helper in DeltaTestImplicits, which allows unit test call sites to still call the old version of StatisticsCollection.recompute and passes None as the catalogTable. This implicit should only be used if the test really only runs against a path-based Delta table and so no catalogTable is present. This is a small refactoring change so existing test coverage is sufficient. ## Does this PR introduce _any_ user-facing changes? No Closes delta-io#2217 GitOrigin-RevId: bc3783d298de7d7ad442ac347042d8fc7820afbe
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which Delta project/connector is this regarding?
Description
This PR replaces all calls to DeltaLog.startTransaction() with calls to DeltaLog.startTransaction(Option[CatalogTable]). This PR is part of #2105 and ensures that transactions have a valid catalogTable attached to them so Uniform can correctly update the table in the catalog.
How was this patch tested?
This is a small refactoring change so existing test coverage is sufficient.
Does this PR introduce any user-facing changes?
No