Optimizing Effectiveness and Efficiency of Software Testing: A Hybrid Approach

pair-wise testing;random testing;pairwise testing;software testing;N-wise testing

Bell, Kera Zakiyah ; Dr. Winser Alexander, Committee Member,Dr. Donald Bitzer, Committee Member,Dr. Laurie Williams, Committee Member,Dr. Mladen Vouk, Committee Chair,Bell, Kera Zakiyah ; Dr. Winser Alexander ; Committee Member ; Dr. Donald Bitzer ; Committee Member ; Dr. Laurie Williams ; Committee Member ; Dr. Mladen Vouk ; Committee Chair

University:North Carolina State University

关键词: pair-wise testing;    random testing;    pairwise testing;    software testing;    N-wise testing;   

Others  :  https://repository.lib.ncsu.edu/bitstream/handle/1840.16/3209/etd.pdf?sequence=1&isAllowed=y

美国|英语

来源: null

【 摘 要 】

The overall goal of software testing is to disclose defects efficiently (i.e. minimal time and cost) and effectively (i.e. maximum faults detected).It takes time to understand what to test, to generate test cases, to execute the test suite and to analyze the results.In a situation where one can parameterize inputs and variables of interest, the cost of generating random operational profile conformant test cases may be acceptable.It is typically O(N*p) where p is the number of parameters and N is the number of test cases.However, this can result in large test suites and may take long to execute.Systematic approaches tend to generate smaller test suites, thus reducing run-time and analysis costs, but may take much longer to generate since they may require a higher-level of initial expertise to develop.Is there a way to maximize benefits of both statistical and systematic approaches together to simultaneously optimize both efficiency and effectiveness?Hybrid approaches combine one or more testing techniques. A particular parameter-based systematic technique of interest is called n-wise testing. It assumes that most of the faults will be found if all or most of the parameter n-tuple values are covered by the tests. The efficiency and effectiveness of a hybrid approach that combines statistical testing with k-wise where 2 <= k < n(k-wise hybrid) testing is explored.Results show that under certain conditions the n-wise hybrid may help maximize efficiency and increase effectiveness beyond statistical testing. Using a hybrid approach seems most effective when the number of values associated with a parameter is very large.The total number of test cases produced by the hybrid technique to detect all n-way defects is approximately vˆ(n-k) times the number of test cases produced by the systematic approach alone where n is the level of testing required to guarantee coverage of all n-way constructs.A potential use of this approach is in testing for failures that may result from a complex combination of interacting parameter values (v), such as those found in security failures, and in testing highly complex network-based systems and workflows in general.