Preview Build Failure: Test Case Refactoring proposal [Work in Progress] #3043
Conversation
| @@ -77,8 +77,8 @@ Aws::String BuildTableName(const char* baseName) | |||
| class TableOperationTest : public ::testing::Test { | |||
|
|
|||
| public: | |||
There was a problem hiding this comment.
These changes allows tests to be run individually using gtest_filter enabling developers to debug case by case, as opposed to having to run all the tests for just even one test to run successfully.
Change is moving from static setup/cleanup to fixture level setup/cleanup in line with how S3 tests have it
|
|
||
| */ | ||
|
|
||
| TEST_F(TableOperationTest, TestCrudOperations2) |
There was a problem hiding this comment.
Demonstration of Proposal for refactoring existing test case to be more robust and resilient .
Check out the forced failure to demonstrate retries for one functional block
| res = res && (hashKeyName.str() == returnedItemCollection[HASH_KEY_NAME].GetS()); | ||
| res = res && (testValueColumn.str() == returnedItemCollection[testValueColumnName].GetS()); | ||
|
|
||
| if(numRetriesLeft > 0) |
There was a problem hiding this comment.
Forced failure to try multiple times till exhaustion leading to success and subsequent GTEST MACRO evaluation
| { | ||
| AWS_LOGSTREAM_TRACE(ALLOCATION_TAG, "TestCrudOperations") | ||
|
|
||
| struct TestContext{ |
There was a problem hiding this comment.
Test specific context. User defined
|
|
||
| deleteRequestIds = requestIds; | ||
|
|
||
| auto putItemReqFunction = [testValueColumnName, crudTestTableName, this, &requestIds]( |
There was a problem hiding this comment.
Example of a function block. Capture whatever is needed and adheres to the function signature of the test planner class
| } | ||
| else | ||
| { | ||
| it = requestIds.erase(it); |
There was a problem hiding this comment.
Note: In this test specific setup, on retries of the function block it only retries elements that had failed.
It is accomplished by maintaining a list which is random node delete friendly.
If a get item request is processed successfully, it's removed .
| } | ||
| AWS_LOGSTREAM_TRACE(ALLOCATION_TAG, "added function with id=" << item.entryId<<" num tries="<<item.retryCount<<" stop on fail="<<item.stopOnFail<<std::endl ); | ||
| //std::cout<<"added function with id=" << item.entryId<<std::endl; |
There was a problem hiding this comment.
print to std outs will be removed before final draft
| } | ||
| private: | ||
| using FunctionSuite = std::list<FunctionBlock>; |
There was a problem hiding this comment.
We can use any container, I used generic iterator interface for this reason. Can make this a template parameter with type traits checks. This is just enhancement
| @@ -100,6 +100,8 @@ class TableOperationTest : public ::testing::Test { | |||
| std::mutex updateItemResultMutex; | |||
| std::condition_variable updateItemResultSemaphore; | |||
|
|
|||
| std::vector<Aws::String> m_tablesCreated; | |||
There was a problem hiding this comment.
Tracks tables created, which gets cleaned up in the fixture destructor invocation
| Aws::UnorderedSet<Aws::String> _uniqueFunctionBlockSet; | ||
| size_t _cursor; | ||
|
|
||
| static const char ALLOCATION_TAG[]; | ||
|
|
||
| //compile time conditional invoking of callable function with different arguments depending upon available context or not in c++ 11 | ||
|
|
||
| using RetryFunctionWrapped = std::function<bool (const RetryFunction_t& callable, const BlockIdentifierType& , size_t) >; |
There was a problem hiding this comment.
Added a way for the Retry Planner class to not mandate context where test case doesn't need one. This is implemented using SFINAE concepts and type traits
|
|
||
| //set global status from block evaluation | ||
| status = status && localResult; | ||
|
|
||
| //if stop on fail after retries, break and retry from start if available | ||
| if(it->stopOnFail && !localResult) | ||
| if(( it->stopStrategy == StopStrategy::STOP_ON_FIRST_FAIL || it->stopStrategy == StopStrategy::STOP_AFTER_ALL_RETRIES ) && !localResult) |
There was a problem hiding this comment.
Added a retry plan stop strategy as needed by use case.
One can choose to continue even on failure, stop after first failures or stop after exhausting all retries.
|
After delibration, we decided on step level retries |
Issue #, if available:
Description of changes:
Check all that applies:
Check which platforms you have built SDK on to verify the correctness of this PR.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.