Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for checking a sentinel file for the cache #1157

Merged
merged 3 commits into from
Apr 24, 2024

Conversation

haoming29
Copy link
Contributor

Closes #1056

@haoming29 haoming29 added enhancement New feature or request cache Issue relating to the cache component labels Apr 24, 2024
@haoming29 haoming29 added this to the v7.8.0 milestone Apr 24, 2024
@haoming29 haoming29 requested a review from turetske April 24, 2024 18:33
docs/parameters.yaml Outdated Show resolved Hide resolved
docs/parameters.yaml Outdated Show resolved Hide resolved
Copy link
Collaborator

@turetske turetske left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add testing to check the cache sentinel location as well? Something similar to what's in origin_test.go?

@@ -492,7 +492,8 @@ from S3 service URL. In this configuration, objects can be accessed at /federati
return originExports, nil
}

func CheckSentinelLocation(exports []OriginExport) (ok bool, err error) {
// Check the sentinel files from Origin.Exports
func CheckOriginSentinelLocations(exports []OriginExport) (ok bool, err error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func CheckOriginSentinelLocations(exports []OriginExport) (ok bool, err error) {
func CheckOriginSentinelLocation(exports []OriginExport) (ok bool, err error) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The s was intentional as we are checking multiple locations for different prefixes

launchers/launcher.go Show resolved Hide resolved
server_utils/origin_test.go Show resolved Hide resolved
server_utils/origin_test.go Show resolved Hide resolved
server_utils/origin_test.go Show resolved Hide resolved
@@ -51,6 +54,21 @@ func CacheServe(ctx context.Context, engine *gin.Engine, egrp *errgroup.Group, m
return nil, err
}

// Check for the sentinel file
if param.Cache_SentinelLocation.IsSet() {
sentinelPath := param.Cache_SentinelLocation.GetString()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems redundant. Can the server_utils function be made more general to check both the cache and origin sentinel locations?

Copy link
Contributor Author

@haoming29 haoming29 Apr 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first draft (not published) was to take advantage of the server_utils function and make them share the same core logic, but I figured the real "core" logic is just a os.Stat call. Given that the error messages have different format for cache and origin, and that origin and cache reads from different parameters, I figured I'd be ok to have a separate function to handle them

@haoming29 haoming29 requested a review from turetske April 24, 2024 19:45
@turetske turetske merged commit d8ddb32 into PelicanPlatform:main Apr 24, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cache Issue relating to the cache component enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a sentinel file for the cache xrootd
2 participants