Flask-Scrapy

A Flask boilerplate for scraping work. I searched web when I needed a scraping(scrapy) boilerplate but didn't find anything useful.So I decided to make a scraping starter project.

Features

• User account sign up, sign in.
• Error handling.
• HTML macros and layout file.
• "Functional" file structure.
• Application factory.
• Static file bundling.
• Logging.
• Scraping
• Downloadable scraped data in csv,json,xls format
• task queue support, celery

Full Stack

• Database: Sqlite
• Web Framework: Flask
• Front End: MDL , Bootstrap
• javascript
• scrapy
• Gunicorn
• Nginx
• Supervisor

Prerequisites

Install all dependency via

pip install -r requirements.txt

Deployment

Follow the guide

• Kickstarting flask on ubuntu

Trying on local machine, just hit

python app.py

Built With

• Flask - The web framework used
• MDL - material design
• Scrapy- Fast scraping framework

Authors

• Apoorv Raj Saxena

License

This project is licensed under the MIT License