A vector table model-based systematic analysis of spectral fault localization techniques

Spectral fault localization (SFL) is an automatic fault-localization technique, which uses risk evaluation formula to rank the risk of fault existence in each program entity after collecting the testing information dynamically. To provide insight into SFL techniques, the evaluation method is an important research topic. In this paper, we present a uniformly systematic investigation framework to evaluate and compare SFL techniques with fixed formulas considering both single-fault and multiple-fault scenarios. Particularly, we design a generic vector table model called VTM as a novel measurement model to thoroughly understand various SFL techniques. By defining different types of faulty statements and investigating suspiciousness factors’ mathematical expression of statements on the basis of VTM, the effectiveness of different SFL techniques could be systematically analyzed and compared. Under the VTM-based evaluation framework, the latest formula D* and the optimal formula O considered before as examples are explored: (1) under a single-fault scenario, O has a better performance than D*, and O is the best SFL technique at present; (2) O shows stable performance and the performance of D* fluctuates in a range under double-fault scenarios: O outperforms D* in three cases, and the performance of O is between the worst performance and the best performance of D* in the other three cases that are less likely to happen; and (3) sample programs are presented to explain such observations. The VTM-based method overcomes the limitations of existing empirical and systematic approaches, which enables a systematic evaluation for SFL techniques with fixed formulas under both single-fault and multiple-fault cases.