DocSearch scraper

This repository holds the code of the DocSearch scraper used to power the hosted version of DocSearch. DocSearch scaper is used as search for the rholang/rholang.github.io implementation.

Pre-requisites

Install Python 3.6. Only this version have no error messages displayed (check that environment variables for python are set). Python has to be included in PATH Var.
Install pipenv
- run command line with admin rights
- $ pip install pipenv

Setup project

Open this project with vscode
$ pipenv install
rename .env.example to .env

Folder structure

.evn
- you need the keys from algolia and the chromedriver path
  - change APPLICATION_ID, API_KEY to yours from algolia
  - change CHROMEDRIVER_PATH to your path (in folder chromedriver_win32)
config/website.json
- config, what css selector to parse on your website and from which website.

Start scrapping

$ pipenv run python ./docsearch run ./configs/website.json