create a virtualenv, activate it, and install requirements in it:

pip install -r requirements.txt

overview
========

data directory
==============
has some canned data snapshots, so far

search.py 
=========

does the search, pulling fetching sets of 4 results, and 
working around google's throttling.  
Output goes into data.txt

TODO: Replace with a real screen scraper, since the API limits to
64 results

    - BeautifulSoup
    - lxml
    - Scrapy


select.py 
=========

parses the repo names out of data.txt and fetches the data on each
repo from github.
Result goes into a dict, which is pickled into pickled.pic

TODO: put the results in a hierarchy, respecting forks



upload.py
=========

TODO: start this, based on some google spreadsheet code I have


show.py
=======

Simply shows what's in the pickle file produced by select.py


gather.py
=========

gathers items into a tree and does some rudimentary scoring.
Takes input from the pickle file, at the moment.

TODO
====
get the 8.5k django search data.
put results to google docs

Resources
=========

Readthedocs.org API - http://read-the-docs.readthedocs.org/en/latest/index.html#developer-documentation
PyPi API - ask Richard Jones if we want a dump of the entire PyPi database. http://wiki.python.org/moin/CheeseShopDev
Github API - http://developer.github.com/