development-policy-category collection

The data and pipeline to build the development-policy-category dataset.

Collection

• collection/source.csv — the list of data sources by organisation, see specification/source
• collection/endpoint.csv — the list of endpoint URLs for the collection, see specification/endpoint
• collection/resource/ — collected resources
• collection/log/ — individual log JSON files, created by the collection process
• collection/log.csv — a collection log assembled from the individual log files, see specification/log
• collection/resource.csv — a list of collected resources, see specification/resource

Updating the collection

We recommend working in virtual environment before installing the python requirements, makerules and other dependencies. Requires Make v4.0 or above.

$ make makerules
$ make init
$ make collect

Building the datasets

The collected files can then be converted into a national dataset:

$ make

Licence

The software in this project is open source and covered by the LICENSE file.

Individual datasets copied into this repository may have specific copyright and licensing, otherwise all content and data in this repository is © Crown copyright and available under the terms of the Open Government 3.0 licence.