Data Reduction for Speeding up the Training of Machine Learning Classifiers
Open AccessThe dramatic increase in dataset volumes available to train machine learning models has led to great advances in machine learning, but at the cost of slowing down training. This dissertation develops and examines data reduction methods to speed up training with little or no drop in accuracy performance, and in some instances with an increase in accuracy. Since many studies have focused on feature selection, but did not adequately consider instance selection, our work focuses on developing novel instance reduction methods, and integrate them with feature reduction methods into integrated holistic reduction approaches. We first examine Simple Random Sample Selection without Replacement for instance selection, integrated with different feature selection methods, such as the Information Gain-based Feature Selection method, and compare its performance with the unintegrated instance selection and feature selection individually, applied at various data reduction rates. Our initial results proved that the integration of instance and feature selection performed much better than instance selection or feature selection alone, in terms of both training speed and even accuracy improvement, encouraging us to pursue integrated reduction further.To that end, we introduce a novel transpose-based instance selection approach, integrate it with different feature selection methods, and investigate the effect of those integrated reduction methods on classifier training time and accuracy performance, for a number of popular classifiers. This approach “tricks” feature selection methods to apply to instance selection, while still applying them for actual feature selection as well. Our results show that this new integrated method leads to greater performance improvement than the one mentioned above, speeding up training significantly without impacting classification accuracy, and in fact, for some classifiers like Naïve Bayes, the accuracy goes up considerably. As data reduction relies on the intuitive insight of reducing data redundancy, and since wavelets have shown great success in that regard in the context of data compression, we also introduce a novel wavelet-based instance selection approach. Integrating that instance reduction approach with various feature selection methods, and examining their performance and comparing that to the performance of the methods examined above, is another contribution of this dissertation.After extensive evaluations of all those approaches and of different design choices for them, we conclude that the wavelet-based data reduction approach is best, followed by the transpose-based approach, both of which far exceed in performance the no-reduction paradigm and the baseline Random Sample Selection approach.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.