Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make scan parallel #33

Open
faern opened this issue Nov 29, 2024 · 0 comments
Open

Make scan parallel #33

faern opened this issue Nov 29, 2024 · 0 comments

Comments

@faern
Copy link
Member

faern commented Nov 29, 2024

unicop is currently single threaded. This makes scanning large code bases a lot slower than it has to be. Make it multi-threaded one way or another.

One suggestion is to keep the directory traversal single threaded in one place. And for each found file the path is sent into some pool of worker threads. These worker threads pick up the message, reads the file and perform the unicode validation on it. The worker thread then returns the result on a channel, instead of directly printing the errors. A special reporter thread reads from the worker threads and prints it.

This should be made in such a way that the output order of errors is deterministic and always the same for a given input code base. The errors should be reported in the same order as the files are traversed on the disk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant