Use ProcessBuilder

[danlooo]

The python client of openEO allows to use normal Python operators resulting in clean and concise code:

bbox = {
        "west": 6.8371137, "south": 50.560007,
        "east": 6.8566699, "north": 50.5647147,
        "crs": "EPSG:4326"
    }
cube = connection.load_collection("TERRASCOPE_S2_TOC_V2",bands=['B04','B08'])\
    .filter_temporal("2017-10-10", "2017-10-30")\
    .filter_bbox(**bbox)

red = cube.band("B04")
nir = cube.band("B08")
ndvi = (nir - red) / (red + nir)

The Operators (+,-,/) are implemented in the class ProcessBuilder.
This is based on the builder design pattern in OOP that is helpful in abstract type conversions.

Can we use this in the Julia client?

[danlooo]

OOP design patterns are obsolete in Julia and can't be directly used. However, multiple dispatch might be useful in processing:

1. Band arithmetic, e.g. cube1.red - cube1.nir: Binary operators of types (band,band) or (band, number)
2. Reducer functions, e.g. cube1.reduce_dimension(reducer = x -> cos(x.mean()) + 1, dimension="t")
3. Apply a function to each element of a data cube, e.g. cube1.apply(x -> x / 255)

with types Band or a placeholder.

[danlooo]

Create and dispatch on a Node variable, analog to ProcessBuilder:

# Build process graph using julia AST eval
import Base: convert, promote, promote_rule
import Base: +, -, *, /, cos, sqrt

struct Node
    op
    children::Vector
end
Node(x::Node) = x
Node(x) = Node("create", [x])
"Create Placeholder node e.g. for user defined functions in reducers and apply functions"
Node() = Node("create", [nothing])

convert(::Type{Node}, x) = Node(x)
convert(::Type{Node}, x::Node) = Node(x)
promote_rule(::Type{Node}, ::Type{T}) where {T<:Any} = Node

cos(x::Node) = Node("cos", [x])
sqrt(x::Node) = Node("sqrt", [x])

+(x::Node, y::Node) = Node("add", [x, y])
*(x::Node, y::Node) = Node("multiply", [x, y])

+(x::Node, y) = +(promote(x, y)...)
*(x::Node, y) = *(promote(x, y)...)

+(x, y::Node) = +(promote(x, y)...)
*(x, y::Node) = *(promote(x, y)...)

resulting in

julia> Node() + cos(0) # cos will be evaluated by julia
Node("add", Node[Node("create", [nothing]), Node("create", [1.0])])

julia> Node() + cos(Node(0)) # cos will be evaluated by Node
Node("add", Node[Node("create", [nothing]), Node("cos", Node[Node("create", [0])])])

julia> Node() + 1 + 2 # julia evals left to right, might result in excess create nodes
Node("add", Node[Node("add", Node[Node("create", [nothing]), Node("create", [1])]), Node("create", [2])])

julia> f = x -> x + 1 * 2
#11 (generic function with 1 method)

julia> f(Node()) # eval lambda syntax e.g. for cube apply or reducer
Node("add", Node[Node("create", [nothing]), Node("create", [2])])

[meggart]

For functional processing patterns I think it would be nice to also rather overload Julias base functions as an alternative to the rather OOP-ish cube1.reduce_dimension(...). E.g. we could define Base.map(f, cube::Node)=cube.apply(f) and provide similar overloads for sum, mean, median etc so that users could run code like

step2 = map(x -> x / 255, step1)
step3 = sum(step2, dims="t")

[danlooo]

Simple arithmetic is added in 8d21293 and branch danlooo/issue22. There was no need for promotion. Instead, the type DataCube was introduced with functions e.g. +(cube::DataCube,number::Real). This type just contains the graph of openEO process calls that will create it. Implicit promotion of e.g. 5 to Node(5) is related to quotation, bypassing the evaluation of Julia code. However, this is very handy, e.g. to reduce 1+1 to 2 directly.

[danlooo]

Closed, see also #23 and #24