Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make BackedUpItems thread safe #8366

Merged

Conversation

sseago
Copy link
Collaborator

@sseago sseago commented Oct 30, 2024

Thank you for contributing to Velero!

Please add a summary of your change

This PR synchronizes access to backupRequest.BackedUpItems with an RWMutex so that multiple ItemBlock processing goroutines will be able to access it.

Does your change fix a particular issue?

Fixes #7148

Please indicate you've done the following:

@sseago sseago force-pushed the synchronise-backedupitems branch from a9342af to ec5d953 Compare October 30, 2024 21:53
@sseago sseago marked this pull request as draft October 30, 2024 21:53
@sseago sseago force-pushed the synchronise-backedupitems branch from ec5d953 to f7f2402 Compare October 30, 2024 23:18
Copy link

codecov bot commented Oct 30, 2024

Codecov Report

Attention: Patch coverage is 93.54839% with 4 lines in your changes missing coverage. Please review.

Project coverage is 59.04%. Comparing base (3c06fc8) to head (015b1e6).
Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
pkg/backup/backup.go 73.33% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8366      +/-   ##
==========================================
+ Coverage   58.98%   59.04%   +0.06%     
==========================================
  Files         368      369       +1     
  Lines       39000    39046      +46     
==========================================
+ Hits        23004    23056      +52     
+ Misses      14532    14527       -5     
+ Partials     1464     1463       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sseago sseago marked this pull request as ready for review October 31, 2024 20:14
@github-actions github-actions bot requested a review from ywk253100 October 31, 2024 20:14
pkg/backup/backed_up_items_map.go Outdated Show resolved Hide resolved
return len(m.BackedUpItems)
}

func (m *backedUpItemsMap) ItemInBackup(key itemKey) bool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with the current names. But just thought of ItemInBackup <-> IsItemBackedup and AddItem <-> MarkItemBackedup

pkg/backup/backed_up_items_map.go Show resolved Hide resolved
@sseago sseago force-pushed the synchronise-backedupitems branch from f7f2402 to d2da99d Compare November 27, 2024 20:59
@sseago
Copy link
Collaborator Author

sseago commented Nov 27, 2024

Minor updates, but I didn't use sync.Map -- see inline comments for more information.

kaovilai
kaovilai previously approved these changes Nov 28, 2024
Copy link
Member

@kaovilai kaovilai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://pkg.go.dev/k8s.io/apimachinery/pkg/util/sets#Set has some of the functions being defined here. If the functions for backedUpItemsMap might grow, it might be wise to use the package there. Otherwise this lgtm.

func (m *backedUpItemsMap) CopyItemList() map[itemKey]struct{} {
m.RLock()
defer m.RUnlock()
returnMap := map[itemKey]struct{}{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
returnMap := map[itemKey]struct{}{}
returnMap := make(map[itemKey]struct{}, len(m.BackedUpItems))

// backedUpItemsMap keeps track of the items already backed up for the current Velero Backup
type backedUpItemsMap struct {
*sync.RWMutex
BackedUpItems map[itemKey]struct{}
Copy link
Contributor

@reasonerjt reasonerjt Nov 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a need to expose "BackedUpItems"? i.e. BackedUpItems -> backedUpItems

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't referenced from outside, so you're right -- no need to expose.

}
}

func (m *backedUpItemsMap) CopyItemList() map[itemKey]struct{} {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not call it "CopyItemMap"? In the func the variable it returns is even called "returnMap"

return returnMap
}

func (m *backedUpItemsMap) ResourceList() map[string][]string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. If the func returns a map, it would be better to call lit "ResourceMap()"
  2. It would be better to add some comments to explain what the key and value look like in the returned map.

return len(m.BackedUpItems)
}

func (m *backedUpItemsMap) ItemInBackup(key itemKey) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call it "Has(key)" but it's trivial...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also like Contains()

@sseago sseago dismissed stale reviews from shubham-pampattiwar and kaovilai via 015b1e6 December 3, 2024 20:23
@sseago sseago force-pushed the synchronise-backedupitems branch from d18cf3c to 015b1e6 Compare December 3, 2024 20:23
Copy link
Contributor

@reasonerjt reasonerjt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks Scott!

@shubham-pampattiwar shubham-pampattiwar merged commit 6c0ed1e into vmware-tanzu:main Dec 4, 2024
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make BackedUpItems thread safe
4 participants