Merge pull request #2 from rl-institut-private/dev

Setup basic structure and files

.gitignore

@@ -127,3 +127,8 @@ dmypy.json
 
 # Pyre type checker
 .pyre/
+
+# Custom stuff
+.DS_Store
+.snakemake
+.idea

CHANGELOG.md

@@ -0,0 +1,15 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is inspired from [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
+and the versioning aim to respect [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+- Setup initial structure and files
+
+### Changed
+
+### Removed
+

PIPELINE.md

@@ -0,0 +1 @@
+# Pipeline docs

README.md

@@ -1,2 +1,51 @@
-# digipipe
-Data and ES pipeline for Digiplan
+<img align="left" width="150" src="https://github.com/rl-institut-private/digipipe/blob/main/docs/img/digiplan-logo.png">
+<img align="right" width="150" src="https://github.com/rl-institut-private/digipipe/blob/main/docs/img/rli_logo.png">
+
+# Digipipe
+
+Pipeline for data and energy system in the Digiplan project.
+
+## Installation 
+
+**Note: Linux only, Windows is currently not supported.**
+
+First, clone via SSH using
+
+    git clone [email protected]:rl-institut-private/digipipe.git /local/path/to/digipipe/
+
+### Install using pip:
+
+Make sure you have Python >= 3.6 installed, let's create a virtual env:
+
+    virtualenv --python=python3.8 venv
+    source venv/bin/activate
+
+Some additional system packages are required, install them by
+
+    apt install gdal-bin python-gdal libspatialindex-dev imagemagick
+
+Notes:
+- Make sure you have GDAL>=3.0 as older versions will not work
+- `imagemagick` is optional and only required for report creation, cf. [PIPELINE.md](PIPELINE.md)
+
+Install package with
+
+    pip install -e /local/path/to/djagora_data/
+
+### Install using conda
+
+Make sure you have conda installed, e.g. miniconda. Then create the env:
+
+    conda create -n digipipe /local/path/to/digipipe/environment.yml
+    conda activate digipipe
+
+## Structure, pipeline and conventions
+
+See [PIPELINE.md](PIPELINE.md)
+
+## Runtime and resources
+
+**TO BE UPDATED**
+
+**Warning:** A full pipeline run takes 10 hours on a Xeon E5-2690 using 14
+cores and requires about 600 GB of disk space.

digipipe/store/3_datasets/TEMPLATE/config.yml

@@ -0,0 +1,3 @@
+"""
+Config file for dataset
+"""

digipipe/store/3_datasets/TEMPLATE/create.py

@@ -0,0 +1,3 @@
+"""
+Process layer: Build new layer from preprocessed data
+"""

environment.yml

@@ -0,0 +1,11 @@
+name: digipipe
+dependencies:
+   - python=3.8
+   - gdal=3.*
+   - python-gdal
+   - libspatialindex
+   - imagemagick
+   - pip
+   - pip:
+      - -e .
+      - -r requirements.txt

requirements.txt

@@ -0,0 +1,15 @@
+pandas
+geopandas
+shapely
+snakemake
+matplotlib
+pyyaml
+descartes
+fiona
+rtree
+plotly
+rasterstats
+networkx
+pygraphviz
+pygments
+rasterio

setup.py

@@ -0,0 +1,51 @@
+from setuptools import setup, find_packages
+from os import path
+
+here = path.abspath(path.dirname(__file__))
+
+# Get the long description from the README file
+with open(path.join(here, "README.md"), encoding="utf-8") as f:
+    long_description = f.read()
+
+with open('requirements.txt') as f:
+    requirements = f.read().splitlines()
+
+setup(
+    name="digipipe",
+    version="0.0.0",
+    description="Pipeline for data and energy system in the Digiplan project.",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url="https://github.com/rl-institut/digipipe/",
+    author="Reiner Lemoine Institut",
+    author_email='[email protected]',
+    license='GNU AGPLv3',
+    classifiers=[  # Optional
+        # How mature is this project? Common values are
+        #   3 - Alpha
+        #   4 - Beta
+        #   5 - Production/Stable
+        "Development Status :: 3 - Alpha",
+        # Indicate who your project is intended for
+        "Intended Audience :: Developers",
+        "Topic :: Software Development :: Build Tools",
+        # Pick your license as you wish
+        "License :: OSI Approved :: AGPL License",
+        # Specify the Python versions you support here. In particular, ensure
+        # that you indicate whether you support Python 2, Python 3 or both.
+        # These classifiers are *not* checked by 'pip install'. See instead
+        # 'python_requires' below.
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.7",
+        "Programming Language :: Python :: 3.8",
+    ],
+    packages=find_packages(),
+    python_requires=">=3, <4",
+    install_requires=requirements,
+    #extras_require={"dev": [], "test": []},  # Optional
+    project_urls={  # Optional
+        "Bug Reports": "https://github.com/rl-institut/digipipe/issues",
+        "Source": "https://github.com/rl-institut/digipipe",
+    },
+)