Structured Expert Judgement Elicitation of Use Error Probabilities for Drug Delivery Device Risk Assessments Open Access
Downloadable ContentDownload PDF
In the pharmaceutical industry, estimating the probability of occurrence for use errors and use-error-causes (here forth referred to as use error probabilities) when developing drug delivery devices is hindered by a lack of data, ultimately limiting the ability to conduct robust usability risk assessments. A lack of reliable data is the result of small sample sizes and challenges simulating actual use environments in simulated use studies, compromising the applicability of observed use error rates. Further, post-market surveillance databases and internal complaint databases are limited in their ability to provide reliable data for product development. Inadequate usability risk assessment hinders drug delivery device manufacturers' understanding of safety and efficacy risks. The current industry and regulatory paradigm with respect to use error probabilities is to de-emphasize them, focusing instead of assessing the severity of harms. However, de-emphasis of use error probabilities is not rooted in a belief that probability estimates inherently lack value. Rather, the status quo is based on the absence of suitable methodologies for estimating use error probabilities. In instances in which data is lacking, engineers and scientist may turn to structured expert judgment elicitation methodologies, in which subjective expert opinions are quantified and aggregated in a scientific manner. This research is a case study in adapting and applying one particular structured expert judgment methodology, Cooke’s Classical model, to human factors experts for estimating use error probabilities for a drug delivery device. Results indicate that a performance-weighted linear pooling of expert judgments significantly outperforms any one expert and an equal-weighted linear pooling. Additionally, this research demonstrates that a performance-weighted linear pooling of expert judgments is statistically accurate, robust to the choice of experts, and robust to choice elicitation questions. Lastly, this research validates the good statistical accuracy of a performance-weighted linear pooling of experts on a new set of use error probabilities, indicating that good expert performance translates to use error probabilities estimates for different devices. Through structured expert judgment elicitation according to Cooke’s Classical model, this research demonstrates that it is possible to reinstall use error probability estimates, with quantified uncertainty, into usability risk assessments for drug delivery devices.