Skip to content

Commit

Permalink
adding info for the cost function
Browse files Browse the repository at this point in the history
  • Loading branch information
zkaoudi committed Feb 12, 2024
1 parent 18ba807 commit 9b004f2
Showing 1 changed file with 14 additions and 19 deletions.
33 changes: 14 additions & 19 deletions docs/guide/add-an-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,30 +26,25 @@ This guide shows the *3 steps* that developers need to follow if they want to ad
We use the Map operator as an example.

## Step 1: Add a Wayang operator
Wayang operators are located under the ```wayang-basic``` in the ```org.apache.wayang.basic.operators``` package.
Wayang operators are located under the ```wayang-basic``` in the ```org.apache.wayang.basic.operators``` package. <br>
An operator needs to extend from one of the following abstract classes: ```UnaryToUnaryOperator```, ```BinaryToUnaryOperator```, ```UnarySource```, ```UnarySink```.<br>
For a unary to unary operator, see for example [here](https://github.com/apache/incubator-wayang/blob/main/wayang-commons/wayang-basic/src/main/java/org/apache/wayang/basic/operators/MapOperator.java).

An operator needs to extend from one of the following abstract classes: ```UnaryToUnaryOperator```, ```BinaryToUnaryOperator```, ```UnarySource```, ```UnarySink```.

For a unary to unary operator, see for example here:
https://github.com/apache/incubator-wayang/blob/main/wayang-commons/wayang-basic/src/main/java/org/apache/wayang/basic/operators/MapOperator.java

For enhanced performance in Wayang, consider adding a cardinality estimator as well.
For enhanced performance in Wayang, consider adding a cardinality estimator by overriding the ```createCardinalityEstimator()``` function as [here](https://github.com/apache/incubator-wayang/blob/main/wayang-commons/wayang-basic/src/main/java/org/apache/wayang/basic/operators/MapOperator.java#L112C43-L112C70).

## Step 2: Add the (platform-specific) execution operators
Execution operators are located under the corresponding module of ```wayang-platforms```. For instance, Java execution operators are located in the ```org.apache.wayang.java.operators``` package of the ```wayang-java``` module.

An execution operator needs to extend from its corresponding Wayang operator and implement the corresponding platform operator interface.

For the above ```MapOperator```, the following is the corresponding ```JavaMapOperator```:
https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/operators/JavaMapOperator.java
Execution operators are located under the corresponding module of ```wayang-platforms```. For instance, Java execution operators are located in the ```org.apache.wayang.java.operators``` package of the ```wayang-java``` module.<br>
An execution operator needs to extend from its corresponding Wayang operator and implement the corresponding platform operator interface.<br>
For the above ```MapOperator```, the following is the corresponding [```JavaMapOperator```](https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/operators/JavaMapOperator.java).

For enhanced performance in Wayang, consider adding a load function as well.
For enhanced performance in Wayang, consider adding a load function as well:<br>
For this you need to overwrite the ```getLoadProfileEstimatorConfigurationKey()``` function and provide the right key that will then be read from a properties file.
For the JavaMapOperator it's: wayang.java.map.load. Then add in the corresponding properties file (e.g., [this](https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-java/src/main/resources/wayang-java-defaults.properties) is for the java executor) the template which is the mathematical formula that represents the cost of this operator and an instantiation of it. See [here](https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-java/src/main/resources/wayang-java-defaults.properties#L25) for the example of the map operator.

## Step 3: Add mappings
Create mappings from the Wayang operator to the platform-specific execution operators. The mappings are located in the corresponding execution module in the ```org.apache.wayang.java.operators``` package.
Create mappings from the Wayang operator to the platform-specific execution operators. <br>
The mappings are located in the corresponding execution module in the ```org.apache.wayang.java.operators``` package.<br>
For the above ```MapOperator``` and ```JavaMapOperator```, see [here](https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/mapping/MapMapping.java).

For the above ```MapOperator``` and ```JavaMapOperator```, see here:
https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/mapping/MapMapping.java
After that you need to declare this mapping in Wayang in the corresponding [```Mappings```](https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/mapping/Mappings.java#L37) class.

After that you need to declare this mapping in the Wayang in the corresponding ```Mappings``` class:
https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/mapping/Mappings.java#L37

0 comments on commit 9b004f2

Please sign in to comment.