Are deep learning models suitable for training with sparse data?



I am training a generative adversarial network (GAN) to generate images given edge histogram descriptor (EHD) features of the image. The EHD features are themselves sparse (meaning they contain a lot of zeroes). While training the generator loss and discriminator loss are reducing very slowly.

Are deep learning models (like GAN) suitable for training with sparse data for one or more of the features in the input or derived through feature extraction?


Posted 2018-08-01T08:23:22.073

Reputation: 480



The problem isn't the GAN but the implementation of its discriminator which is typically a convolutional neural network (CNN). CNNs have trouble with sparse data. They require dense data to learn well. There are ways to work around this. See the following for some ideas:

Brian O'Donnell

Posted 2018-08-01T08:23:22.073

Reputation: 746