Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improve ] introduce asyncio or thread parallelism in WikiCFPScrapper.py #5

Open
rajatkb opened this issue Feb 22, 2020 · 4 comments
Open

[Improve ] introduce asyncio or thread parallelism in WikiCFPScrapper.py #5

rajatkb opened this issue Feb 22, 2020 · 4 comments
Labels
enhancement New feature or request gssoc20 GSSOC label for gscco20 tag hard GSSOC label for beginner tag
Projects
Scrapper-Service

Comments

@rajatkb
Copy link
Owner

rajatkb commented Feb 22, 2020

The current implementation is made keeping in mind future prospect of parallelism. Can make the implementation leverage Python threading for faster processing.
@rajatkb rajatkb added enhancement New feature or request medium GSSOC label for beginner tag labels Feb 22, 2020
@rajatkb rajatkb added this to To do in Scrapper-Service via automation Feb 22, 2020
@rajatkb rajatkb added gssoc20 GSSOC label for gscco20 tag hard GSSOC label for beginner tag and removed medium GSSOC label for beginner tag labels Feb 23, 2020
@rajatkb rajatkb changed the title Introduce asyncio or thread parallelism in WikiCFPScrapper.py [Improve ] introduce asyncio or thread parallelism in WikiCFPScrapper.py Feb 28, 2020
@vipuldcoder
Copy link

I am interested with the issue.

@abhishekgupta368
Copy link
Contributor

I am Interested, can you assign me.

@rajatkb rajatkb linked a pull request Mar 12, 2020 that will close this issue
added_multiprocessing #26
Merged
@rajatkb
Copy link
Owner Author

rajatkb commented Mar 12, 2020

@abhishekgupta368 , are you comfortable with using Python asyncIO package ? Also have you gone through the WikiCFP scrapper, The issue would require a thorough understanding of these to tackle. Revert back to me on the same.

@abhishekgupta368
Copy link
Contributor

I think I should leave it then.

Scrapper-Service automation moved this from To do to Done Mar 13, 2020
@rajatkb rajatkb removed a link to a pull request Mar 13, 2020
added_multiprocessing #26
Merged
@rajatkb rajatkb reopened this Mar 13, 2020
Scrapper-Service automation moved this from Done to To do Mar 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request gssoc20 GSSOC label for gscco20 tag hard GSSOC label for beginner tag
Projects
Scrapper-Service
  
To do
Milestone
No milestone
Development

No branches or pull requests
3 participants
@rajatkb @vipuldcoder @abhishekgupta368