A comparison of Assessment of Software Reliability Using Classical and Bayesian Methods for two commonly used software reliability models Open Access
Downloadable ContentDownload PDF
Software reliability growth models (SRGMs) provide a framework to analyze software failures by using past failure data to predict future software reliability or to estimate current software reliability. Most current models use a classical approach to infer parameters and have shown limitations in prediction accuracy. The Bayesian approach has been claimed to be more successful than classical approaches in certain situations, as it allows the incorporation of prior information into models. In this study, we evaluated software reliability prediction in 20 sets of data failures using either the maximum likelihood (classical) or Bayesian approaches to estimate parameters of two Non-homogenous Poisson models: the Musa basic time execution (MBTE) and the Musa-Okumoto logarithmic models. To validate the model parameter estimation, the parameters were estimated using 10, 30, 50, 70 and 90 percent of the data. While with the Bayesian approach, parameters were estimated at all tested percentages, with the classical approach, parameters were only estimated when at least 50 percent of the data were used for half of the datasets. The performance of the Bayesian and classical approaches were compared for 10 datasets using a novel methodology in which the weighted average of eight performance measures is determined as a single permanent value. The permanent value was lower with the Bayesian approach than with the classical approach for six out of 10 failure datasets using the MBTE model, indicating that the Bayesian approach outperformed the classical method for these datasets. In contrast, with the Musa Okumoto model, the Bayesian approach performed better than the classical only for four of 10 datasets. A sensitivity analysis was performed to evaluate the influence of the prior distribution on the model performance when the parameters were estimated using the Bayesian inference. We analyzed two datasets for which the classical approach outperformed the Bayesian approach for both the MBTE and Musa-Okumoto models. Interestingly, by changing the prior distribution limits, the Bayesian method outperformed the classical method either with the MBTE or with the Musa-Okumoto models. Overall the results of this study indicate that while the Bayesian approach was not superior to maximum likelihood, the Bayesian approach was clearly advantageous as it could accurately estimate model parameters using a small percentage of failure data. As the performance of the Bayesian approach was improved by changing prior distributions, future studies should explore ways to accurately define prior distributions.