Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential race condition when checking file size #70

Open
CorentinB opened this issue Jan 30, 2025 · 2 comments
Open

Potential race condition when checking file size #70

CorentinB opened this issue Jan 30, 2025 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@CorentinB
Copy link
Owner

CorentinB commented Jan 30, 2025

panic: open ./warcs/VIDEO-20250130132906660-00005-crawler01.warc.gz.open: no such file or directory

goroutine 32124 [running]:
github.com/CorentinB/warc.isFileSizeExceeded({0xc16c7faf40?, 0xc046388be0?}, 0x408f400000000000)
        /root/go/pkg/mod/github.com/!corentin!b/[email protected]/utils.go:284 +0xf5
github.com/CorentinB/warc.recordWriter(0xc04ce75440, 0xc04a67f7a0, 0xc041956a80, 0xc046388bf0)
        /root/go/pkg/mod/github.com/!corentin!b/[email protected]/warc.go:148 +0x559
created by github.com/CorentinB/warc.(*RotatorSettings).NewWARCRotator in goroutine 1
        /root/go/pkg/mod/github.com/!corentin!b/[email protected]/warc.go:72 +0x8a```
@CorentinB CorentinB added the bug Something isn't working label Jan 30, 2025
@CorentinB CorentinB self-assigned this Jan 30, 2025
@vbanos
Copy link
Contributor

vbanos commented Feb 2, 2025

I suggest the following solution:
Pass the file path instead of *os.File. This ensures the function operates on a fresh file descriptor each time, reducing potential race conditions.
Also, very important: before checking the file path, check if the file exists. If not, just return false

@CorentinB
Copy link
Owner Author

I suggest the following solution: Pass the file path instead of *os.File. This ensures the function operates on a fresh file descriptor each time, reducing potential race conditions. Also, very important: before checking the file path, check if the file exists. If not, just return false

The problem with that is that it makes the lib open and close files, A LOT. If we keep and os.File and don't have this problem, it avoids that, there shouldn't be a race even with os.File.. :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants