Testcase to test the api quota limit #165
Conversation
|
Also the table reset test is missing a manipulate state method. Is this in another PR somewhere? I thought I remembered someone talking about building that out. |
|
|
||
| # run check mode | ||
| # Run a check job using orchestrator (discovery) | ||
| with self.assertRaises(Exception) as ex: |
There was a problem hiding this comment.
per run did not raise exception on last commit. could the check job use less than 1 percent quota?
FAIL: test_api_per_run_quota (test_salesforce_api_per_run_quota.SFAPIQuota)
Run the tap in check mode and verify it returns the error for quota limit reached. If not, proceed to sync mode.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/project/tests/test_salesforce_api_per_run_quota.py", line 60, in test_api_per_run_quota
found_catalogs = self.run_and_verify_check_mode( conn_id)
AssertionError: Exception not raised
2023-08-30 14:32:00,401 TapTester - INFO ***** Test Results *****
----------------------------------------------------------------------
Ran 1 tests in 107s
There was a problem hiding this comment.
We can't have less than 1%, couldn't send fraction amount. This is very tricky, let's discuss.
There was a problem hiding this comment.
If it is not guaranteed to hit the per run quota in check mode can we do a "try / catch" for check mode and then use the normal "assertRaises" for sync mode? That way if the quota is hit during check mode we can catch the error and verify it and if it is not hit until sync mode we can also catch it there. If that works I'd use the same pattern for both tests.
https://jira.talendforge.org/browse/TDL-23972 - I also remember a conversation regarding this, created this card |
|
|
||
| conn_id = connections.ensure_connection(self) | ||
|
|
||
| # run check mode |
There was a problem hiding this comment.
| # run check mode |
There was a problem hiding this comment.
Are you in favor of keeping this line?
|
|
||
| # run check mode | ||
| # Run a check job using orchestrator (discovery) | ||
| with self.assertRaises(Exception) as ex: |
There was a problem hiding this comment.
If it is not guaranteed to hit the per run quota in check mode can we do a "try / catch" for check mode and then use the normal "assertRaises" for sync mode? That way if the quota is hit during check mode we can catch the error and verify it and if it is not hit until sync mode we can also catch it there. If that works I'd use the same pattern for both tests.
| # Run a check job using orchestrator (discovery) | ||
| try: | ||
| found_catalogs = self.run_and_verify_check_mode(conn_id) | ||
| except Exception as ex: |
There was a problem hiding this comment.
| except Exception as ex: | |
| except Exception as ex: | |
| self.assertIn(expected_per_run_error, str(ex.exception)) | |
| raise ex |
There was a problem hiding this comment.
If the try catch raises the exception we can still do the assertion on the exception I think.
|
|
||
| conn_id = connections.ensure_connection(self) | ||
|
|
||
| # run check mode |
There was a problem hiding this comment.
Are you in favor of keeping this line?
Description of change
https://jira.talendforge.org/browse/TDL-23431
Run the sync job and make sure the per-run-limit is applied.
Split from feature https://jira.talendforge.org/browse/TDL-23349
QA steps
Risks
Rollback steps