A Quantitative Comparison of the Effects of Modeling Approaches on System Verification Using a Controlled Challenge Problem
Open AccessTo reduce program risks, engineering methods capitalizing on modeling and machine assistance have been extensively investigated within systems engineering (and more specifically requirements engineering) literature over the past 20 years. However, there are few quantitative comparisons between model-based approaches and legacy document-centric approaches. Studies have shown that the lack of data regarding improvements of modeling has decelerated the adoption of model-based practices. To help address this gap, a screening experiment was conducted to compare the effects of modeling on an engineer’s ability to determine if a system has met its originating requirements. First, a notional acquisition program was created based on an unmanned aerial system, including originating requirements as well as both document and model-based design artifacts. Requirements were captured in both a traditional requirements document and a goal-oriented requirements engineering model. System implementation data was captured in both traditional document artifacts and a SysML model. Participants of varying experience levels used randomized combinations of document and model-based approaches to determine whether the notional system met its requirements. The experiment measured the review duration and accuracy, permitting analysis of the effects of model-based approaches for both requirements and system implementation data. The results of the experiment showed that use of a requirements model did not statistically affect the review. A system implementation model was shown to improve novice participants’ reviews but did not statistically affect experienced participants. The results of this research should inform future research on the use of models, particularly the return on the modeling investment.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.