Skip to content

The idea is to collect existing jobs directly from Web3 companies and aggregate them for future use and analysis

License

Notifications You must be signed in to change notification settings

crypto-jobs-fyi/crawler

Repository files navigation

crypto-jobs-crawler

The idea is to build a dashboard with crypto jobs plus some index number to track industry health. Plus you can search for a job as well :)

support the project

If you find this project useful please donate ETH/ERC-20 to 0x589a0d87d600a6c6faa34c491c9e779f434bc51d

roadmap

  • automate scraping positions from most popular HR platforms
  • enable simple way of displaying these positions in browser
  • enable storage of data as it needed for index calculations
  • build better UI with simple search feature
  • ...

how to use?

Given that you have Python 3.5+ and pip installed just run pip install -r requirements.txt Crawler uses just 1 none standard libs:

  • selenium
  • pip install -r requirements.txt Next just run crawler.py

Docker

  • docker build --tag scrap:latest .
  • docker run --rm -it -v ${PWD}:/data docker.io/library/scrap:latest

UI staff

Let's try GitHub pages here: https://crypto-jobs-fyi.github.io/web/ Locally just run python3 -m http.server 8000 and open localhost:8000 in your browser.

What we will watch -> work in progress

wip:

not only crypto:

not yet crypto:

to be fixed:

About

The idea is to collect existing jobs directly from Web3 companies and aggregate them for future use and analysis

Resources

License

Stars

Watchers

Forks