- Python 3
- Library: Scrapy, Selenium
- Database: MySQL, PostgresSQL (update soon)
Step 1: Clone
git clone https://github.com/ltdungg/Crawl-Data-From-Tiki.git
Step 2: Go to work directory
cd Crawl-Data-From-Tiki\TikiCrawler
Step 3: Install requirements
pip install requirements.txt
Step 4: Create vitural environment
python -m venv venv
Step 5: Activate vitural environment
venv/Scripts/activate
Step 6: Crawling
-O: overwrite
-o: append
python -m scrapy crawl main -O output.csv