A Data-Driven Framework for Missing Data Imputation
Open AccessMultiple imputation (MI) is an effective and flexible tool to handle missing data in large databases. By imputing missing data several times, MI generates multiple complete data sets that would allow different data users to perform their own substantive analyses with no missing observations. When conducting MI, the imputation model should be compatible with the substantive analysis model. In other words, the imputation model should include all relationships that are going to be investigated in the substantive analysis model, such as nonlinearities and interactions. Otherwise, the relationships that exist may not be identified in the subsequent analysis, generating biased estimates of parameters and leading to misleading inference. Unfortunately, traditional MI methods, such as multivariate imputation by chained equations (MICE), are built on parametric imputation models. With dozens or even hundreds of variables, as is often the case in large databases, identifying interactive and nonlinear relations and encoding them in imputation models can be a daunting task with no guarantee of success. Unlike parametric models, machine learning techniques (MLTs) are model-free methods, and thus provide flexibility for missing data imputation. MLTs use algorithms that automatically learn from all data to detect statistical dependencies in observations without being explicitly programmed where to look.In this study, we proposed a data-driven framework (MICE-ML) that replaces parametric imputation models with MLTs within the MICE algorithm. Specifically, MLTs including neural networks and random forests are integrated into MICE. Through a simulation study, we demonstrated the robustness of MLTs in capturing the complex relationships among variables for imputation. We further imputed missing data in the National Inpatient Sample (NIS) database using MICE-ML and developed an R package (miceml) that can be used to implement MICE-ML.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.