-
Notifications
You must be signed in to change notification settings - Fork 6
Planner Algorithms
Typically, modern planners use lots of online planning and replanning techniques since they are expected to operate while the system is executing the plan. For example, a self driving car might create a plan to get from point A to point B. The self driving car would then have utilize different techniques (neural networks, machine learning, etc) to process data as it drives in order to determine which actions to take. The self driving system is constantly replanning and evaluating the environment around it to make the best decision for the goal at hand (get to point B in this case).
For testing, the system under test is expected to work. Therefore, many current reseach topics in AI may not be directly applicable since the created plan generated is expected to work. However, this does not discount potential applications of these technologies in this realm. Further exploration is needed.
In this documents a plan is the test case.
- Progresses forward throughout the state space
- Generate shorest (therefore fastest) possible tests
- Always finds the optimal path
- Guarenteed solution
- Simple to implement
- Very slow (essentially unusable) for large state spaces
- This is the currently used algorithm for all planning
- Progresses forward throughout the state space
- Uses heuristic prioritizes nodes that have new actions
- Generate tests is for larger systems fairly quickly
- Can be a lot faster than regular search
- Guarenteed solution
- Does very well for systems that have new actions that chain together
- Very slow if new actions do not chain together
- Simply reverts back to slow searching when can't find anything new
- Does not find the optimal path
- Progresses forward throughout the state space
- Produce a specific set of plans based on options
- Generate tests for smaller systems to ensure it meets certain standards
- Good for small state spaces
- Can generate tests easily for different coverage criteria
- Example: Create all plans of length 5 or less
- Guarenteed solution
- Very slow (essentially unusable) for large state spaces
- Use multiple workers (threads, other computers) to search through the state space
- Use multiple heuristics for the workers
- Generate tests faster
- Most likely faster
- Can use multiple approaches on same state space
- Potentially high complexity
- NodeJS only has experimental support for threads in version 10
- Randomly generate plans
- Directed by options, if desired
- Example: Length, tags, other metadata
- Take very different paths through the system
- Quickly make an infiinite number of valid tests
- Extremely simple
- Produce many different paths through the system
- May not meet coverage criteria
- No guarenteed solution
- Create new plans based on existing plans
- Currently referred to by the misnomer "longer tests algorithm"
- Filter tests to more suit certain use cases needs with options
- Produce "better" plans now that knowledge has been gained about the system under test
- Achieve a complex set of goals more easily
- Example: Simplify existing plans, produce plans for less stringent coverage criteria
- Requires test plans to have already been generated
- Depending on options, a solution may not be guarenteed (think harder about if this is true)
- Can get very complex depending on supported options
- Mutant: an invalid test
- Create a mutant plan using seed plans using a set of mutators
- Run the mutant against the system and expect it to fail
- Easily create negative style tests
- An automated way to generate negative tests
- Very quick to generate tests
- This less accurate than a true negative test since it only expects a failure to occur and not a specific failure
- Dependent on coming up with good mutators
- Create a new valid plan using seed tests cases (possibly human curated)
- The plan is created using a set of mutators
- Run the "genetically modified" plan against the system and expect it to pass
- Use a scoring system to determine if the "genetically modified" plan is better
- Create higher quality tests in an automated way
- Cheap way to generate similar test cases
- Create better and better tests (if scoring system is good)
- Dependent on coming up with good mutators
- Dependent on a scoring system for creating "better" plans
- Expensive if the scoring system depends on running the test
- Use a dataset to learn how to create a plan
- Machine learning, neural networks, etc
- Very high quality tests
- Potentially very high quality plans based on useful data
- In sync with current research and may be able to utilize already build solutions (Cloud APIs, etc)
- Must have good data
- Lots of technical expertise requried