A Bayesian Approach for Predicting Verification Test Defects and Schedule Uncertainty
Open AccessIn the medical device industry new devices are brought to market after verification and validation (V&V;) testing, pending the Food and Drug Administration’s (FDA) approval. There are complex devices that require extensive bench testing of both software and hardware. Verification test efforts consume a substantial portion of project schedules and introduce risks specific to duration uncertainty because of the probability of test issues or defects (Maatta et al., 2009). This often directly conflicts with time to market pressures as global competition increases and product lifecycles compress. Organizations are operating in increasingly competitive markets and one way to increase profit and market share is to launch new, quality products before competitors. This is often accomplished by shortening design lifecycles (Griffin, 1997; Perttula A, 2004). There is a gap in this area of research, an investigation and predictive model built to study the cause-and-effect relationship between verification test defects and project schedule delay has not been publicized. Most studies involved with planning and optimizing V&V; focus on the cost benefit relationship between testing quantity and field failures associated with reliability (Abdel Azeem et al., 2014; Ahmed & Chateauneuf, 2014; Maatta et al., 2009; Mobin et al., 2019a, 2019b; Selvy et al., 2014). Since time to market is an ever-present issue for new product development (NPD), one overlooked risk in project scheduling is verification test duration. With highly regulated industries, like medical device, test activities are subject to a large amount of scrutiny by regulating bodies, and therefore have years of legacy documentation with descriptions, logs, and time durations of every test defect, observation, root cause analysis, and resolution. With so much readily available data in issue tracking systems such as Jira, building a Bayesian Belief Network (BBN) predictive model with informed priors, proved to be an effective way to mitigate verification test risk. Jira data was filtered into categories to construct conditional probabilities for test defect occurrence using Bayes’ Theorem. In combination with a carefully structured directed graph, these probabilities were linked to generate a predictive model for test defects and resulting schedule delay associated with the defect type. Sensitivity analysis showed that test defects were most influenced by Tester Experience and Procedure Typos respectively. The model predicted test defects with 73% accuracy and schedule delay with 99% accuracy. The BBN methods were 13% more accurate than the critical path method (CPM).
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.