A Bayesian Framework for Software Regression Testing

Software Engineering;Regression Testing;Test case prioritization;Bayesian Networks;Electrical and Computer Engineering (Software Engineering)

Mir arabbaygi, Siavash

University of Waterloo

关键词: Software Engineering;    Regression Testing;    Test case prioritization;    Bayesian Networks;    Electrical and Computer Engineering (Software Engineering);   

Others  :  https://uwspace.uwaterloo.ca/bitstream/10012/3936/1/thesis_final%20submission_2.pdf

瑞士|英语

来源: UWSPACE Waterloo Institutional Repository

【 摘 要 】

Software maintenance reportedly accounts for much of the total cost associatedwith developing software. These costs occur because modifying software is a highlyerror-prone task. Changing software to correct faults or add new functionalitycan cause existing functionality to regress, introducing new faults. To avoid suchdefects, one can re-test software after modifications, a task commonly known asregression testing.Regression testing typically involves the re-execution of test cases developed forprevious versions. Re-running all existing test cases, however, is often costly andsometimes even infeasible due to time and resource constraints. Re-running testcases that do not exercise changed or change-impacted parts of the program carriesextra cost and gives no benefit. The research community has thus sought ways tooptimize regression testing by lowering the cost of test re-execution while preservingits effectiveness. To this end, researchers have proposed selecting a subset of testcases according to a variety of criteria (test case selection) and reordering test casesfor execution to maximize a score function (test case prioritization).This dissertation presents a novel framework for optimizing regression testingactivities, based on a probabilistic view of regression testing. The proposed frameworkis built around predicting the probability that each test case finds faults in theregression testing phase, and optimizing the test suites accordingly. To predict suchprobabilities, we model regression testing using a Bayesian Network (BN), a powerfulprobabilistic tool for modeling uncertainty in systems. We build this model usinginformation measured directly from the software system. Our proposed frameworkbuilds upon the existing research in this area in many ways. First, our frameworkincorporates different information extracted from software into one model, whichhelps reduce uncertainty by using more of the available information, and enablesbetter modeling of the system. Moreover, our framework provides flexibility byenabling a choice of which sources of information to use. Research in softwaremeasurement has proven that dealing with different systems requires different techniquesand hence requires such flexibility. Using the proposed framework, engineerscan customize their regression testing techniques to fit the characteristics of theirsystems using measurements most appropriate to their environment.We evaluate the performance of our proposed BN-based framework empirically.Although the framework can help both test case selection and prioritization, wepropose using it primarily as a prioritization technique. We therefore compare ourtechnique against other prioritization techniques from the literature. Our empiricalevaluation examines a variety of objects and fault types. The results show that theproposed framework can outperform other techniques on some cases and performscomparably on the others.In sum, this thesis introduces a novel Bayesian framework for optimizing regressiontesting and shows that the proposed framework can help testers improve thecost effectiveness of their regression testing tasks.