Evaluation of Test Process Improvement approaches: An industrial case study

Abstract

Context: Test Process Improvement (TPI) approaches are frameworks or models that guide software development organizations to investigate, assess and improve their software testing process. Objectives: We extend existing work in the area of Test Process Improvement by identifying available approaches and by evaluating them in regards to their characteristics. Furthermore, two selected approaches are evaluated with respect to their content and assessment results. Methods: In the first part of this study we use a systematic literature review to identify the existing TPI approaches which are then used in the second part of the study. The second part of the study is an industrial case study in which two TPI approaches are applied in an industrial setting. Results: We contribute in providing (1) a complete, in our opinion, list of 16 existing TPI approaches and their characteristics, (2) a detailed comparison of the content and the results of the two applied approaches (TPI Next and TMMi) and (3) experience in applying them in industry. As a result of this research we found that the content as well as the assessment results of the two approaches are similar to a great extent. Conclusions: Numerous Test Process Improvement approaches are available, but not all are generally applicable for industry. One major difference between available approaches is their model representation. Even though, the applied approaches generally show strong similarities, differences in the assessment results are noticeable due to their different model representations.