SUBMARINE-1071. Fix pysubmarine test of tensorflow2

### What is this PR for?
<!-- A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://submarine.apache.org/contribution/contributions.html
-->
Due to package version problem, we need to downgrade tensorflow-estimator to 2.6.0 and upgrade numpy to 1.19.2.
Also I have refactored the workflow of python-sdk test to make it more clear.

### What type of PR is it?
Bug Fix

### Todos
* [x] - Fix test

### What is the Jira issue?
<!-- * Open an issue on Jira https://issues.apache.org/jira/browse/SUBMARINE/
* Put link here, and add [SUBMARINE-*Jira number*] in PR title, eg. `SUBMARINE-23. PR title`
-->
https://issues.apache.org/jira/projects/SUBMARINE/issues/SUBMARINE-1071?filter=allopenissues
### How should this be tested?
<!--
* First time? Setup Travis CI as described on https://submarine.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
-->
By Github action
### Screenshots (if appropriate)

### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No

Author: featherchen <[email protected]>

Signed-off-by: Kevin <[email protected]>

Closes #792 from featherchen/SUBMARINE-1071 and squashes the following commits:

68fb460 [featherchen] change job name
7d231ae [featherchen] refactor
3df75c2 [featherchen] refactor
2d4dc79 [featherchen] refactor
25c7776 [featherchen] refactor
e231e51 [featherchen] add -e
e1348d5 [featherchen] echo
01afc4c [featherchen] refactor python workflow of CI/CD
c556f43 [featherchen] refactor python workflow of CI/CD
0108c24 [featherchen] refactor python workflow of CI/CD
bdc7545 [featherchen] refactor python workflow of CI/CD
e342e00 [featherchen] refactor python workflow of CI/CD
bb316cd [featherchen] refactor python workflow of CI/CD
a9f307b [featherchen] refactor python workflow of CI/CD
042ae7c [featherchen] downgrade tensorflow-estimator to 2.6.0
4185c5e [featherchen] downgrade tensorflow-estimator to 2.6.0
1ad8e8b [featherchen] downgrade tensorflow-estimator to 2.6.0

.github/workflows/python.yml

@@ -19,33 +19,43 @@ name: python-sdk
 on: [push, pull_request]
 
 jobs:
+  check-style:
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v2
+      - name: Install dependencies
+        run: |
+          pip install --upgrade pip  
+          pip install -r ./dev-support/style-check/python/lint-requirements.txt
+          pip install -r ./dev-support/style-check/python/mypy-requirements.txt
+      - name: Check python sdk code style
+        run: ./dev-support/style-check/python/lint.sh
+
   unit:
     runs-on: ubuntu-latest
     timeout-minutes: 30
     strategy:
       matrix:
         python-version: [3.6, 3.7]
-        tf-version: [1.14.0, 1.15.0, 2.6.0]
+        tf-version: [1.15.0, 2.6.0]
       fail-fast: false
     steps:
       - uses: actions/checkout@v2
       - name: Setup python environment
         uses: actions/setup-python@v2
         with:
           python-version: ${{ matrix.python-version }}
-      - name: Install dependencies
+      - name: Install requirements
         run: |
-          pip install --upgrade pip
-          pip install --no-cache-dir tensorflow==${{ matrix.tf-version }}
-          pip install --no-cache-dir torch==1.5.0
-          pip install --no-cache-dir tensorflow-addons
-          pip install --no-cache-dir tf_slim
-          pip install --no-cache-dir ./submarine-sdk/pysubmarine/.
+          pip install --upgrade pip  
           pip install -r ./submarine-sdk/pysubmarine/github-actions/test-requirements.txt
-          pip install -r ./dev-support/style-check/python/lint-requirements.txt
-          pip install -r ./dev-support/style-check/python/mypy-requirements.txt
-      - name: Check python sdk code style
-        run: ./dev-support/style-check/python/lint.sh
+      - name: Install pysubmarine with tf1 and pytorch
+        if: matrix.tf-version == '1.15.0'
+        run: pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf,pytorch]
+      - name: Install pysubmarine with tf2 and pytorch
+        if: matrix.tf-version == '2.6.0'
+        run: pip install --no-cache-dir -e ./submarine-sdk/pysubmarine/.[tf2,pytorch]
       - name: Run unit test
         run: pytest --cov=submarine -vs -m "not e2e"
 

submarine-sdk/pysubmarine/setup.py

@@ -28,7 +28,7 @@
     packages=find_packages(exclude=["tests", "tests.*"]),
     install_requires=[
         "six>=1.10.0",
-        "numpy==1.18.5",
+        "numpy==1.19.2",
         "pandas",
         "sqlalchemy>=1.4.0",
         "sqlparse",
@@ -42,8 +42,13 @@
         "boto3>=1.17.58",
     ],
     extras_require={
-        "tf": ["tensorflow>=1.14.0,<2.0.0"],
-        "tf2": ["tensorflow==2.6.0", "tf_slim==1.1.0", "tensorflow-addons==0.14.0"],
+        "tf": ["tensorflow==1.15.0"],
+        "tf2": [
+            "tensorflow==2.6.0",
+            "tf_slim==1.1.0",
+            "tensorflow-addons==0.14.0",
+            "tensorflow-estimator==2.6.0",
+        ],
         "pytorch": ["torch>=1.5.0", "torchvision>=0.6.0"],
     },
     classifiers=[

submarine-sdk/pysubmarine/tests/ml/tensorflow/model/test_base_tf_model.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow.model.base_tf_model import BaseTFModel
-
 
 @pytest.mark.skipif(tf.__version__ >= "2.0.0", reason="requires tf1")
 def test_create_base_tf_model():
+    from submarine.ml.tensorflow.model.base_tf_model import BaseTFModel
+
     params = {"learning rate": 0.05}
     with pytest.raises(AssertionError, match="Does not define any input parameters"):
         BaseTFModel(params)

submarine-sdk/pysubmarine/tests/ml/tensorflow/model/test_ccpm.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow.model import CCPM
-
 
 @pytest.mark.skipif(tf.__version__ >= "2.0.0", reason="requires tf1")
 def test_run_ccpm(get_model_param):
+    from submarine.ml.tensorflow.model import CCPM
+
     params = get_model_param
 
     model = CCPM(model_params=params)

submarine-sdk/pysubmarine/tests/ml/tensorflow/model/test_deepfm.py

@@ -16,11 +16,12 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow.model import DeepFM
-
 
 @pytest.mark.skipif(tf.__version__ >= "2.0.0", reason="requires tf1")
 def test_run_deepfm(get_model_param):
+
+    from submarine.ml.tensorflow.model import DeepFM
+
     params = get_model_param
 
     model = DeepFM(model_params=params)

submarine-sdk/pysubmarine/tests/ml/tensorflow/model/test_fm.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow.model import FM
-
 
 @pytest.mark.skipif(tf.__version__ >= "2.0.0", reason="requires tf1")
 def test_run_fm(get_model_param):
+    from submarine.ml.tensorflow.model import FM
+
     params = get_model_param
 
     model = FM(model_params=params)

submarine-sdk/pysubmarine/tests/ml/tensorflow/model/test_nfm.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow.model import NFM
-
 
 @pytest.mark.skipif(tf.__version__ >= "2.0.0", reason="requires tf1")
 def test_run_nfm(get_model_param):
+    from submarine.ml.tensorflow.model import NFM
+
     params = get_model_param
 
     model = NFM(model_params=params)

submarine-sdk/pysubmarine/tests/ml/tensorflow/test_optimizer.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow.optimizer import get_optimizer
-
 
 @pytest.mark.skipif(tf.__version__ >= "2.0.0", reason="requires tf1")
 def test_get_optimizer():
+    from submarine.ml.tensorflow.optimizer import get_optimizer
+
     optimizer_keys = ["adam", "adagrad", "momentum", "ftrl"]
     invalid_optimizer_keys = ["adddam"]
 

submarine-sdk/pysubmarine/tests/ml/tensorflow_v2/model/test_base_tf_model.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow_v2.model.base_tf_model import BaseTFModel
-
 
 @pytest.mark.skipif(tf.__version__ < "2.0.0", reason="requires tf2")
 def test_create_base_tf_model():
+    from submarine.ml.tensorflow_v2.model.base_tf_model import BaseTFModel
+
     params = {"learning rate": 0.05}
     with pytest.raises(AssertionError, match="Does not define any input parameters"):
         BaseTFModel(params)

submarine-sdk/pysubmarine/tests/ml/tensorflow_v2/model/test_ccpm.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow_v2.model import CCPM
-
 
 @pytest.mark.skipif(tf.__version__ < "2.0.0", reason="requires tf2")
 def test_run_ccpm(get_model_param):
+    from submarine.ml.tensorflow_v2.model import CCPM
+
     params = get_model_param
 
     model = CCPM(model_params=params)

submarine-sdk/pysubmarine/tests/ml/tensorflow_v2/model/test_deepfm.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow_v2.model import DeepFM
-
 
 @pytest.mark.skipif(tf.__version__ < "2.0.0", reason="requires tf2")
 def test_run_deepfm(get_model_param):
+    from submarine.ml.tensorflow_v2.model import DeepFM
+
     params = get_model_param
 
     model = DeepFM(model_params=params)

submarine-sdk/pysubmarine/tests/ml/tensorflow_v2/model/test_fm.py

@@ -16,11 +16,12 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow_v2.model import FM
-
 
 @pytest.mark.skipif(tf.__version__ < "2.0.0", reason="requires tf2")
 def test_run_fm(get_model_param):
+
+    from submarine.ml.tensorflow_v2.model import FM
+
     params = get_model_param
 
     model = FM(model_params=params)

submarine-sdk/pysubmarine/tests/ml/tensorflow_v2/model/test_nfm.py

@@ -16,11 +16,11 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow_v2.model import NFM
-
 
 @pytest.mark.skipif(tf.__version__ < "2.0.0", reason="requires tf2")
 def test_run_nfm(get_model_param):
+    from submarine.ml.tensorflow_v2.model import NFM
+
     params = get_model_param
 
     model = NFM(model_params=params)

submarine-sdk/pysubmarine/tests/ml/tensorflow_v2/test_optimizer.py

@@ -16,11 +16,12 @@
 import pytest
 import tensorflow as tf
 
-from submarine.ml.tensorflow_v2.optimizer import get_optimizer
-
 
 @pytest.mark.skipif(tf.__version__ < "2.0.0", reason="requires tf2")
 def test_get_optimizer():
+
+    from submarine.ml.tensorflow_v2.optimizer import get_optimizer
+
     optimizer_keys = ["adam", "adagrad", "momentum", "ftrl"]
     invalid_optimizer_keys = ["adddam"]