Sentinel 2 Super-Resolution processing block

Introduction

This is a state of the art processing block using a Convolutional Neural Network algorithm to derive higher resolution images from existing lower resolution images using Sentinel-2 datasets as input. The code is adapted from https://github.com/lanha/DSen2, our thanks go to the authors of the original code base and the corresponding paper.

Another goal of this project is to help users setting up their TensorFlow based algorithms on UP42. The block functionality and performed processing steps are described in more detail in the UP42 documentation: S2 Super-Resolution.

Block Input: Sentinel 2 L1C product.

Block Output: GeoTIFF file.

Requirements

This example requires the Mac or Ubuntu bash. In order to bring this example block or your own custom block to the UP42 platform the following tools are required:

• UP42 account - Sign up for free!
• Python 3.7
• A virtual environment manager e.g. virtualenvwrapper
• git
• docker engine
• GNU make

Instructions

The following step-by-step instructions will guide you through setting up, dockerizing and pushing the example custom block to UP42.

Clone the repository

git clone https://github.com/up42/s2-superresolution.git
cd s2-superresolution

Then navigate to the folder via cd s2-superresolution.

Installing the required libraries

First create a new virtual environment called up42-supres, for example by using virtualenvwrapper:

mkvirtualenv --python=$(which python3.7) up42-supres

Activate the new environment:

workon up42-supres

Install the necessary Python libraries via:

make install

Testing the block locally

Before uploading the block to the UP42 platform, we encourage you to run the following local tests and validations to ensure that the block works as expected, conforms to the UP42 specifications and could be successfully applied in a UP42 workflow.

Run the unit tests

By successfully running the implemented Python unit tests you can ensure that the block processing functionality works as expected. This project uses pytest for testing, which was installed in the previous step. Run the unit tests via:

make test

Validate the manifest

Then test if the block manifest is valid. The UP42manifest.json file contains the block capabilities. They define what kind of data a block accepts and provides, which parameters can be used with the block etc. See the UP42 block capabilities documentation. Validate the manifest via:

make validate

Run the end-to-end test

In order to run the final end-to-end (e2e) test the block code needs to be dockerized (put in a container that later on would be uploaded to UP42). The end-to-end test makes sure the block's output actually conforms to the platform's requirements. To run the e2e tests it is necessary to have gsutil installed which is part of the Google Cloud SDK.

First build the docker image locally.

make build

Run the e2e tests with:

make e2e

Pushing the block to the UP42 platform

First login to the UP42 docker registry. <USERNAME> needs to be replaced by your UP42 username, which is the email address you use on the UP42 website.

docker login -u=<USERNAME> http://registry.up42.com

# Example:
docker login [email protected] http://registry.up42.com

In order to push the block to the UP42 platform, you need to build the block Docker container with your UP42 USER-ID. To get your USER-ID, go to the UP42 custom-blocks menu. Click on "PUSH a BLOCK to THE PLATFORM" and copy your USERID from the command shown on the last line at "Push the image to the UP42 Docker registry". The USERID will look similar to this: 63uayd50-z2h1-3461-38zq-1739481rjwia

Replace <User-ID> with your UP42 User-ID.

docker build . -t registry.up42.com/<USER-ID>/s2-superresolution:1.0 --build-arg manifest="$(cat UP42Manifest.json)"

# Example:
docker build . -t registry.up42.com/6760d08e-54e3-4f1c-b22e-6ba605ec7592/s2-superresolution:1.0 --build-arg manifest="$(cat UP42Manifest.json)"

Now you can finally push the image to the UP42 docker registry. Replace <User-ID> with your UP42 User-ID.

docker push registry.up42.com/<USER-ID>/s2-superresolution:1.0

# Example:
docker push registry.up42.com/6760d08e-54e3-4f1c-b22e-6ba605ec7592/s2-superresolution:1.0

Success! The block will now appear in the UP42 custom blocks menu menu and can be selected under the Custom blocks tab when building a workflow.

Optional: Updating an existing custom block

If you want to update a custom block on UP42, you need to build & push the Docker container with a higher version number.

Support, questions and suggestions

Open a github issue in this repository; we are happy to answer your questions!