Understanding, detecting, and repairing performance bugs

Performance bugs

Nistor, Adrian

关键词: Performance bugs;   

Others  :  https://www.ideals.illinois.edu/bitstream/handle/2142/49706/Adrian_Nistor.pdf?sequence=1&isAllowed=y

美国|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

Software performance is critical for how end-users perceive thequality of software products.Performance bugs---programming errorsthat cause performance degradation—lead to poor user experience andlow system throughput.Despite advances in profiling techniques,performance bugs still escape in production runs.There are two keyreasons why performance bugs are not effectively detected duringin-house testing.First, there is little available data about howperformance bugs are discovered, reported, and fixed in practice.Such data is required when designing effective techniques foraddressing performance bugs.Second, the current techniques fordetecting performance bugs detect only slow computation and do notaddress other important parts of the testing process, such asautomated oracles or bug fixing.This dissertation makes three contributions.The first contributionis a study of how performance bugs are discovered, reported todevelopers, and fixed by developers, and how these results comparewith the results for non-performance bugs.The study considersperformance and non-performance bugs from three popular code bases:Eclipse JDT, Eclipse SWT, and Mozilla.First, we find little evidencethat fixing performance bugs has a higher chance to introduce newfunctional bugs than fixing non-performance bugs, which implies thatdevelopers may not need to be over-concerned about fixing performancebugs.Second, although fixing performance bugs is about aserror-prone as fixing non-performance bugs, fixing performance bugs ismore difficult than fixing non-performance bugs, indicating thatdevelopers need better tool support for fixing performance bugs andtesting performance bug patches.Third, unlike many non-performancebugs, a large percentage of performance bugs are discovered throughcode reasoning, not through users observing the negative effects ofthe bugs (e.g., performance degradation) or through profiling.Theresult suggests that techniques to help developers reason aboutperformance, better test oracles, and better profiling techniques areneeded for discovering performance bugs.The second contribution is TODDLER, a novel automated oracle forperformance bugs, which enables testing for performance bugs to usethe well established and automated process of testing for functionalbugs.TODDLER reports code loops whose computation has repetitive andpartially similar memory-access patterns across loop iterations.Suchrepetitive work is likely unnecessary and can be done faster.TODDLERwas implemented for Java and evaluated on 9 popular Java codebases.The experiments with 11 previously known, real-world performance bugsshow that TODDLER finds these bugs with a higher accuracy than thestandard Java profiler.TODDLER also found 42 new bugs in six Javaprojects: Ant, Google Core Libraries, JUnit, Apache Collections, JDK,and JFreeChart.Based on the corresponding bug reports, developers sofar fixed 10 bugs and confirmed 6 more as real bugs.The third contribution is LULLABY, a novel static technique thatdetects and fixes performance bugs that have non-intrusive fixeslikely to be adopted by developers.Each performance bug detected byLULLABY is associated with a loop and a condition.When the conditionbecomes true during the loop execution, all the remaining computationperformed by the loop is wasted.Developers typically fix suchperformance bugs because these bugs waste computation in loops andhave non-intrusive fixes: when some condition becomes truedynamically, just break out of the loop.Given a program, LULLABYdetects such bugs statically and gives developers a potentialsourcelevel fix for each bug.LULLABY was evaluated on real-worldapplications, including 11 popular Java applications (e.g., Groovy,Log4J, Lucene, Struts, Tomcat, etc) and 4 widely used C/C++applications (Chromium, GCC, Mozilla, and MySQL).LULLABY finds 61new performance bugs in the Java applications and 89 new performancebugs in the C/C++ applications.Based on the corresponding bugreports, developers so far have fixed 51 and 65 performance bugs inthe Java and C/C++ applications, respectively.Most of the remainingbugs are still under consideration by developers.