Establishing Trust in U.S. Department of Defense Operator-in-the-Loop Federations-of-Models Open Access
Downloadable ContentDownload PDF
Virtual environments involving Federations-of-Models (FoM) and Operators-in-the-Loop (OITL) offer a unique venue for investigating the engineering trade-space available for System-of-Systems (SoS) designs. Use of an OITL FoM environment early in the design process (i.e., during concept development, technology development and early design development phases) can provide insights into systems integration challenges and human factors issues early enough to inform design decisions. By combining some of the benefits of rapid prototyping and large-force exercises, simulations conducted using an OITL FoM can also provide initial indications of military utility to inform funding and "portfolio management" decisions within the U.S. Department of Defense (DoD). The technical and organizational complexity associated with SoS-based design development within DoD adds presents decisionmakers with significant uncertainty. This translates into an increasing dependency on simulation results and introduces a corresponding necessity to establish well-informed trust in all aspects of modeling and simulation used to characterize proposed, future warfighting capabilities. OITL FoM environments applied in the investigation of future human-centric SoS-based capabilities, however, challenge the verification, validation and accreditation (VV&A;) practices broadly applied to establish the credibility of smaller-scale modeling and simulation (M&S;) employed at the platform, weapon-system or weapon level. Specifically, absence of an extant simuland (i.e., a mature, fielded system) for some models within the SoS can complicate the identification of an acceptable referent against which a model can be validated. Furthermore, the introduction of human operators in the virtual environment places the problem of validation in a category of "special cases" that often receive only cursory treatment in professional journals and DoD guidance on M&S.; The research reflected in this dissertation further explores the systems engineering challenges associated with SoS-based capability development, with a specific focus on the use of large-scale interactive simulations. Preferred and alternate approaches for establishing trust in modeling and simulation are considered in the context of an OITL FoM to identify the desirable attributes of a new and improved approach. A method is developed that adds rigor to informal V&V; practices through an emphasis on the conceptual model of the SoS-based capability and attention to the pedigree of both the models in the federation and the operators. Results are presented in an intuitive format that facilitates identification of areas where credibility might be easily (or necessarily) enhanced, engenders trust, and remains compatible with current DoD guidance for the VV&A; of M&S;.