Improvements to the Process of Measuring System Architecture Properties Through Systems Engineering Data Creation, Experimentation, and Simulation
Open AccessThere is a growing importance to measuring system architecture properties, such as complexity, modularity or other “illities,” but after an extensive review of Engineering Design and Systems Engineering (EDSE) literature, there is no leading or agreed-upon methodology to measure these properties. Consequently, there is little agreed upon theory about these phenomena, and I contend that the difficulty in measuring properties and in representing system architectures contributes to the difficulty in developing theory and must be addressed. When trying to measure and theorize, researchers are partially unable to communicate and develop our findings on architecture properties into concrete theory due to the amount of variation in how we measure system architecture properties. There are disparate datasets of engineering systems, represented in a variety of ways with discretion within the different methods, and abundant measures of a given phenomenon or construct with often limited construct validity. Collectively, this variance in the measurement process leads to vastly different outcomes of measurement related to a system architecture phenomenon, and an inability to reconcile findings. This dissertation addresses the variability in the measurement process and comprises of three major research thrusts to address those concerns. First, I created a large set of system architecture representations, functional descriptions, and a database structure for data from the Astrobee Open Innovation Field Experiment, a challenge series with 17 unique engineering challenges with high degrees of replication for a variety of intentionally architected systems as well as engineer/designer data. This enabled us to provide information about thousands of solvers, more than 250 engineering designs, and over 120 system architecture and functional descriptions of systems to the research community. Secondly, I explored how system architecture representation affected the measurement of architecture properties using real-world engineering data. This work found that a slight variation in how we represented system architectures led to changes in measurement value and rank of a system within a set and that system architecture representation and that that design strategies interact with representation and measurement. Thirdly, I synthesized the literature on complexity into a few commonly held beliefs. I found none of the representative measures used consistently captured those beliefs. They were tested via a benchmarking approach that leveraged synthetic system architectures to assess the construct validity of measures of engineering complexity. Furthermore, I found systematic bias in how complexity conceptualization impacted complexity measurement, where certain system architectures are complex for one viewpoint but not another. Combined, these three research thrusts contribute to the systems engineering literature on measuring and theorizing about system architecture properties. We provided data to community to provide more opportunities for research and validation with regards to system architecture, problem decomposition, open innovation, and the relationships between problems, solvers, and their solutions. We characterized how sensitive measures are to architecture representation and developed new ways to represent cyber-physical system architectures. We also created a benchmarking technique for researchers to assess the construct validity of their tools they wish to measure complexity with. These contributions build a better foundation for future systems engineering research.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.