Some Research Progress on Generative Adversarial Networks
Open AccessGenerative adversarial networks (GANs) have been widely studied during recent years. GANs are generative models based on the deep learning framework to generate artificial data. A large number of successful applications of GANs have been demonstrated in various research areas. The Jenson-Shannon Divergence (JSD) is an essential foundation of GANs. However, to our knowledge, when the distribution of real data cannot be precisely formulated, the robustness of GANs-related JSD has not been well addressed. In this study, we propose an extension of JSD: robust JSD, to address this robustness concern. We have investigated some theoretical properties related to the robust JSD. Specifically, we have obtained some concrete mathematical results based on several exponential family distributions, the first-order and the second-order Taylor approximations, and Kullback-Leibler divergence (KLD). We show that, under certain conditions, during a common numerical computing procedure for the objective min-max optimization, the generator related distribution and the classifier related distribution can move closer to the true distribution after each iterative step. Furthermore, we have studied a weighted version of robust JSD and a geometric mean alternative to robust JSD. We have conducted some related simulation studies to demonstrate our theoretical results. Moreover, we have developed a speed-up by re-initialization algorithm (SUBRIA) to accelerate the numerical convergence. Under SUBRIA, based on several exponential family distributions, we show that the min-max game of robust JSD is equivalent to the maximum likelihood estimations (MLE). Lastly, an application is provided to illustrate robust JSD on MNIST data set.For the frameworks of GANs, tree structure has been developed with deep neural networks a few years ago, but the fusion between the tree structure and GANs has not been studied a lot. In our study, we propose a deep regression tree based GAN (DRT-based-GAN), a new algorithm to develop a vanilla GAN based on a tree structure. A DRT-based-GAN is constructed by designing a generator based on a deep regression tree (DRT). For a forest, each single tree is defined to generate one simulated observation. All simulated observations are collected from multiple trees, which forms a deep regression forest based GAN (DRF-based-GAN). The well-known back-propagation-based optimization has been extended for this framework. We use Python-Pytorch to implement our algorithm. Furthermore, we have also considered the Least Squares GAN (LSGAN) and the Wasserstein GAN (WGAN) for our framework. We have conducted simulation studies to evaluate the performance of our algorithms. The advantages of our algorithm have been demonstrated by the simulation results. Moreover, we have discussed the relationship between our framework and Gaussian mixture models (GMM). Furthermore, we have discussed the connection between our framework and the Polya tree (PT). We show that, under reasonable assumptions, for a general one-dimensional PT, an equivalent DRT-based-GAN can be constructed. Some simulation studies have been conducted to evaluate theoretical results. A real data set has also been used to illustrate our framework.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.