TabPy v2.0.0 (#441)

* added anvoa to supported pre-deployed models in tabpy (#350) * added anvoa to supported pre-deployed models in tabpy * fixed pep8 issue * fixed md * Add Ctrl+C handler (#348) * Add Ctrl+C handler * Fix unit tests warnings for genson * Add test to increase code coverage * Add * Change default from 10Mb to 100Mb for request size * Increase code coverage * Increase code coverage * Convert buffer size to int * Add Ctrl+C test * Delete test added to the wrong folder * Update CHANGELOG * Update test_app.py * Remove dead code * Don't count coverage for multiline expressions * Add test case for invalid protocol * Add test case for _check_endpoint_name * Remove dead code * Fix vulnerabilities found by LGTM (#361) * Fix vulnerabilities found by LGTM * Fix test failures * Dev improvements (#384) * Fix flake8 warnings * Merge from master * Fix pycodestyle * Fix more flake8 warnings * Fix tests to pass again * Create test_coveralls_codestyle.yml (#382) * Use github actions * Documentation improvements (#385) * Delete .travis.yml * Fix Ctrl+C failing on extra parameter in signal handler * Remove outdated documentation for how to configure connection * tabpy.py to use docopt * Update tabpy-user with docopt * Update CHANGELOG * Fix code style * Change regex, add remove method, and edit qeury_timeout (#375) * Added Client.remove method to delete deployed model * Fix bug for query_timeout types * Update CHANGELOG * Fix missing script result/return bug * Fix github workflow for push * Code improvement: app._parse_config (#391) * Update app.py * v0.8.10 (#392) * Collect test coverage with scrutinizer instead of coveralls * Restore coverage collecting with coveralls * Update scrutinizer settings * Add support for Python 3.8 * Fix static page and add unit test for it * Delete obsolete test * Dev cov (#394) * Collect test coverage with scrutinizer instead of coveralls * Restore coverage collecting with coveralls * Update scrutinizer settings * Add support for Python 3.8 * Fix static page and add unit test for it * Delete obsolete test * Restore scrutinizer configuration * Linting as separate build step * Restore scrutinizer configuration * Update .scrutinizer.yml * Update .scrutinizer.yml * Restore scrutinizer configuration * Update pull_request.yml * Code style improvements * Code style improvements * Code style improvements * Add coverall workflow * Initial checkin, working tests. * Made common base class for server info tests. * pep8 checks. * Added documentation for TABPY_AUTH_INFO * Fix spacing. * Refactor config parsing to allow custom parsers. (#412) * Refactor config parsing to allow custom parsers. * Fix pep8 * Update version and changelog. * Changed default for tabpy_auth_info to a boolean. * Secure info (#414) * Update README.md * Doc update (#402) * Fixed broken link * Linked to install doc. * Dev fix spelling (#408) * Add spelling fix workflow * Refactor config parsing to allow custom parsers. * Fix pep8 * Update version and changelog. * Changed default for tabpy_auth_info to a boolean. * Remove configuration for securing the info API and make it secure by default. * Revert "Merge branch 'master' into secureInfo" This reverts commit 3688561, reversing changes made to a4acc65. * Removed auth config from uni test. Co-authored-by: nmannheimer <[email protected]> Co-authored-by: Oleksandr Golovatyi <[email protected]> * Secure info (#417) * Update README.md * Doc update (#402) * Fixed broken link * Linked to install doc. * Dev fix spelling (#408) * Add spelling fix workflow * Refactor config parsing to allow custom parsers. * Fix pep8 * Update version and changelog. * Changed default for tabpy_auth_info to a boolean. * Remove configuration for securing the info API and make it secure by default. * Revert "Merge branch 'master' into secureInfo" This reverts commit 3688561, reversing changes made to a4acc65. * Removed auth config from uni test. * Removed example for removed setting. * Remove unused example config. Co-authored-by: nmannheimer <[email protected]> Co-authored-by: Oleksandr Golovatyi <[email protected]> * Clean up API documentation (#420) * Update README.md * Doc update (#402) * Fixed broken link * Linked to install doc. * Dev fix spelling (#408) * Add spelling fix workflow * Update LICENSE update to Tableau Software LLC * v 1.1.0: Secure /info with auth (#415) - Authorization is now required for the /info API method. This method did not check authentication previously. This change is backwards compatible with Tableau clients. - Improved config parsing flexibility. Previously the TABPY_EVALUATE_TIMEOUT setting would be set to a default if tabpy couldn't parse the value. Now it will throw an exception at startup. * Clean up API documentation * Clean up API documentation * Clean up API documentation Co-authored-by: nmannheimer <[email protected]> Co-authored-by: lriggs <[email protected]> Co-authored-by: Olek Golovatyi <[email protected]> * Code improvements (#431) * Rename tabpy_server and tabpy_tools to server and tools (breaking change) * fix flake8 warnings * Clean up code to reduce number of conditions * Remove pypi publishing instructions - those are Tableau specific * Restore tabpy_tools and tabpy_server names * Restore tabpy_tools and tabpy_server names * Restore tabpy_tools and tabpy_server names * Unit and integration tests passing * Update .gitignore * do not track settings.json for VSCode * Fix server -> tabpy_server * more cleaning for tabpy_server names * make "python setup.py test" work * add coverage module as required * delete tests node for scrutinizer run * Update postman collection * remove print from error handling code (#439) * remove print from error handling code * remove & for linux cmd Co-authored-by: Olek Golovatyi <[email protected]> * Return HTTP 400 status when receiving a request with authentication credentials and authN is not configured (#440) * added 400 bad request response to event when authorization is not set up but user sends username and password * added unit tests to test what happens when there are credentials, but no authentication is required * renamed not_authorized flag to authentication_error flag * changed line formatting * changed the way auth error is handled * Version to 2.0.0 * Version to 2.0.0 * Fix codystyle warnings * Restore scrutinizer settings * Fix codestyle Co-authored-by: sbabayan <[email protected]> Co-authored-by: ogolovatyi <[email protected]> Co-authored-by: Brennan Bugbee <[email protected]> Co-authored-by: Logan Riggs <[email protected]> Co-authored-by: Olek Golovatyi <[email protected]> Co-authored-by: nmannheimer <[email protected]> Co-authored-by: harold-xi <[email protected]>
tableau · Aug 14, 2020 · e8b0703 · e8b0703
1 parent 903b167
commit e8b0703
Show file tree

Hide file tree

Showing 19 changed files with 356 additions and 114 deletions.
diff --git a/.scrutinizer.yml b/.scrutinizer.yml
@@ -25,6 +25,7 @@ build:
           -
             command: pylint-run
             use_website_config: true
+    tests: true
 checks:
   python:
     code_rating: true

diff --git a/CHANGELOG b/CHANGELOG
@@ -1,6 +1,16 @@
 # Changelog
 
-## v1.2.0
+## v2.0.0
+
+### Breaking changes
+
+- TabPy fails with 400 when it is not configure for authentication
+  but credentials are provided by client.
+
+### Bug fixes
+
+- When TabPy is running with no console attached it is not failing
+  with 500 when trying to respond with 401 status.
 
 ### Improvements
 

diff --git a/misc/TabPy.postman_collection.json b/misc/TabPy.postman_collection.json
@@ -6,19 +6,16 @@
 	},
 	"item": [
 		{
-			"name": "{{endpoint}}/info",
+			"name": "{{host}}:{{port}}/info",
 			"request": {
 				"method": "GET",
 				"header": [],
-				"body": {
-					"mode": "raw",
-					"raw": ""
-				},
 				"url": {
-					"raw": "{{endpoint}}/info",
+					"raw": "{{host}}:{{port}}/info",
 					"host": [
-						"{{endpoint}}"
+						"{{host}}"
 					],
+					"port": "{{port}}",
 					"path": [
 						"info"
 					]
@@ -27,28 +24,69 @@
 			"response": []
 		},
 		{
-			"name": "{{endpoint}}/evaluate",
+			"name": "{{host}}:{{port}}/evaluate",
 			"request": {
+				"auth": {
+					"type": "basic",
+					"basic": [
+						{
+							"key": "password",
+							"value": "P@ssw0rd",
+							"type": "string"
+						},
+						{
+							"key": "username",
+							"value": "user1",
+							"type": "string"
+						}
+					]
+				},
 				"method": "POST",
 				"header": [
 					{
 						"key": "Content-Type",
 						"name": "Content-Type",
 						"value": "application/json",
 						"type": "text"
+					},
+					{
+						"key": "TabPy-Client",
+						"value": "Postman for manual testing",
+						"type": "text"
+					},
+					{
+						"key": "TabPy-User",
+						"value": "ogolovatyi",
+						"type": "text"
 					}
 				],
 				"body": {
 					"mode": "raw",
-					"raw": "{\n\t\"data\": \n\t{ \n\t\t\"_arg1\" : [1, 2, 3], \n\t\t\"_arg2\" : [3, -1, 5]\n\t},\n\t\"script\": \n\t\"res = []\\nfor i in range(len(_arg1)):\\n  res.append(_arg1[i] * _arg2[i])\\nreturn res\"\n}\n"
+					"raw": "{\n\t\"data\": \n\t{ \n\t\t\"_arg1\" : [1, 2, 3], \n\t\t\"_arg2\" : [3, -1, 5]\n\t},\n\t\"script\": \n\t\"return [x + y for x, y in zip(_arg1, _arg2)]\"\n}\n",
+					"options": {
+						"raw": {}
+					}
 				},
 				"url": {
-					"raw": "{{endpoint}}/evaluate",
+					"raw": "{{host}}:{{port}}/evaluate",
 					"host": [
-						"{{endpoint}}"
+						"{{host}}"
 					],
+					"port": "{{port}}",
 					"path": [
 						"evaluate"
+					],
+					"query": [
+						{
+							"key": "TabPy-Client",
+							"value": "Postman for Manual Testing",
+							"disabled": true
+						},
+						{
+							"key": "TabPy-User",
+							"value": "ogolovatyi",
+							"disabled": true
+						}
 					]
 				}
 			},
@@ -59,15 +97,12 @@
 			"request": {
 				"method": "GET",
 				"header": [],
-				"body": {
-					"mode": "raw",
-					"raw": ""
-				},
 				"url": {
-					"raw": "{{endpoint}}/status",
+					"raw": "{{host}}:{{port}}/status",
 					"host": [
-						"{{endpoint}}"
+						"{{host}}"
 					],
+					"port": "{{port}}",
 					"path": [
 						"status"
 					]
@@ -76,19 +111,16 @@
 			"response": []
 		},
 		{
-			"name": "{{endpoint}}/endpoints",
+			"name": "{{host}}:{{port}}/endpoints",
 			"request": {
 				"method": "GET",
 				"header": [],
-				"body": {
-					"mode": "raw",
-					"raw": ""
-				},
 				"url": {
-					"raw": "{{endpoint}}/endpoints",
+					"raw": "{{host}}:{{port}}/endpoints",
 					"host": [
-						"{{endpoint}}"
+						"{{host}}"
 					],
+					"port": "{{port}}",
 					"path": [
 						"endpoints"
 					]
@@ -97,7 +129,7 @@
 			"response": []
 		},
 		{
-			"name": "{{endpoint}}/query/add",
+			"name": "{{host}}:{{port}}/query/model_name",
 			"request": {
 				"method": "POST",
 				"header": [
@@ -113,35 +145,33 @@
 					"raw": "{\r\n  \"data\": {\r\n    \"x\": [\r\n      6.35,\r\n      6.4,\r\n      6.65,\r\n      8.6,\r\n      8.9,\r\n      9,\r\n      9.1\r\n    ],\r\n    \"y\": [\r\n      1.95,\r\n      1.95,\r\n      2.05,\r\n      3.05,\r\n      3.05,\r\n      3.1,\r\n      3.15\r\n    ]\r\n  }\r\n}"
 				},
 				"url": {
-					"raw": "{{endpoint}}/query/add",
+					"raw": "{{host}}:{{port}}/query/model_name",
 					"host": [
-						"{{endpoint}}"
+						"{{host}}"
 					],
+					"port": "{{port}}",
 					"path": [
 						"query",
-						"add"
+						"model_name"
 					]
 				}
 			},
 			"response": []
 		},
 		{
-			"name": "{{endpoint}}/endpoints/add",
+			"name": "{{host}}:{{port}}/endpoints/model_name",
 			"request": {
 				"method": "GET",
 				"header": [],
-				"body": {
-					"mode": "raw",
-					"raw": ""
-				},
 				"url": {
-					"raw": "{{endpoint}}/endpoints/add",
+					"raw": "{{host}}:{{port}}/endpoints/model_name",
 					"host": [
-						"{{endpoint}}"
+						"{{host}}"
 					],
+					"port": "{{port}}",
 					"path": [
 						"endpoints",
-						"add"
+						"model_name"
 					]
 				}
 			},

diff --git a/tabpy/VERSION b/tabpy/VERSION
@@ -1 +1 @@
-1.2.0
+2.0.0
diff --git a/tabpy/tabpy_server/common/default.conf b/tabpy/tabpy_server/common/default.conf
@@ -47,7 +47,7 @@ propagete=0
 
 [handler_rootHandler]
 class=StreamHandler
-level=DEBUG
+level=INFO
 formatter=rootFormatter
 args=(sys.stdout,)
 

diff --git a/tabpy/tabpy_server/handlers/base_handler.py b/tabpy/tabpy_server/handlers/base_handler.py
@@ -6,6 +6,7 @@
 import tornado.web
 from tabpy.tabpy_server.app.SettingsParameters import SettingsParameters
 from tabpy.tabpy_server.handlers.util import hash_password
+from tabpy.tabpy_server.handlers.util import AuthErrorStates
 import uuid
 
 
@@ -132,16 +133,12 @@ def initialize(self, app):
             app.settings[SettingsParameters.LogRequestContext]
         )
         self.logger.log(logging.DEBUG, "Checking if need to handle authentication")
-        self.not_authorized = not self.handle_authentication("v1")
+        self.auth_error = self.handle_authentication("v1")
 
     def error_out(self, code, log_message, info=None):
         self.set_status(code)
         self.write(json.dumps({"message": log_message, "info": info or {}}))
 
-        # We want to duplicate error message in console for
-        # loggers are misconfigured or causing the failure
-        # themselves
-        print(info)
         self.logger.log(
             logging.ERROR,
             'Responding with status={}, message="{}", info="{}"'.format(
@@ -364,7 +361,7 @@ def _validate_credentials(self, method) -> bool:
         )
         return False
 
-    def handle_authentication(self, api_version) -> bool:
+    def handle_authentication(self, api_version):
         """
         If authentication feature is configured checks provided
         credentials.
@@ -376,27 +373,36 @@ def handle_authentication(self, api_version) -> bool:
 
         Returns
         -------
-        bool
-            True if authentication is not required.
-            True if authentication is required and valid
-            credentials provided.
-            False otherwise.
+        String
+            None if authentication is not required and username and password are None.
+            None if authentication is required and valid credentials provided.
+            NotAuthorized if authenication is required and credentials are incorrect.
+            NotRequired if authentication is not required but credentials are provided.
         """
         self.logger.log(logging.DEBUG, "Handling authentication")
         found, method = self._get_auth_method(api_version)
         if not found:
-            return False
+            return AuthErrorStates.NotAuthorized
 
         if method == "":
-            # Do not validate credentials
-            return True
+            if not self._get_basic_auth_credentials():
+                self.logger.log(logging.DEBUG,
+                                "authentication not required, username and password are none")
+                return AuthErrorStates.NONE
+            else:
+                self.logger.log(logging.DEBUG,
+                                "authentication not required, username and password are not none")
+                return AuthErrorStates.NotRequired
 
         if not self._get_credentials(method):
-            return False
+            return AuthErrorStates.NotAuthorized
 
-        return self._validate_credentials(method)
+        if not self._validate_credentials(method):
+            return AuthErrorStates.NotAuthorized
 
-    def should_fail_with_not_authorized(self):
+        return AuthErrorStates.NONE
+
+    def should_fail_with_auth_error(self):
         """
         Checks if authentication is required:
         - if it is not returns false, None
@@ -405,21 +411,35 @@ def should_fail_with_not_authorized(self):
         Returns
         -------
         bool
-            False if authentication is not required or is
-            required and validation for credentials passes.
-            True if validation for credentials failed.
+            False if authentication is not required and username
+            and password is None or isrequired and validation
+            for credentials passes.
+            True if validation for credentials failed or
+            if authentication is not required and username and password
+            fields are not empty.
         """
-        return self.not_authorized
+        return self.auth_error
 
-    def fail_with_not_authorized(self):
+    def fail_with_auth_error(self):
         """
-        Prepares server 401 response.
+        Prepares server 401 response and server 400 response depending
+        on the value of the self.auth_error flag
         """
-        self.logger.log(logging.ERROR, "Failing with 401 for unauthorized request")
-        self.set_status(401)
-        self.set_header("WWW-Authenticate", f'Basic realm="{self.tabpy_state.name}"')
-        self.error_out(
-            401,
-            info="Unauthorized request.",
-            log_message="Invalid credentials provided.",
-        )
+        if self.auth_error == AuthErrorStates.NotAuthorized:
+            self.logger.log(logging.ERROR, "Failing with 401 for unauthorized request")
+            self.set_status(401)
+            self.set_header("WWW-Authenticate", f'Basic realm="{self.tabpy_state.name}"')
+            self.error_out(
+                401,
+                info="Unauthorized request.",
+                log_message="Invalid credentials provided.",
+            )
+        else:
+            self.logger.log(logging.ERROR, "Failing with 400 for Bad Request")
+            self.set_status(400)
+            self.set_header("WWW-Authenticate", f'Basic realm="{self.tabpy_state.name}"')
+            self.error_out(
+                400,
+                info="Bad request.",
+                log_message="Username or Password provided when authentication not available",
+            )