How to generate the original image from feature set?



We all know that using CNN, or even simpler functions, like CLD or EHD, we can generate a set of features out of images.

Is there any ways or approaches that given a set of features, we can somehow generate a corase version of the original image that was given as input? Maybe a gray-scale version with visible objects inside? If so, what features do we need?

Tina J

Posted 2019-08-22T19:50:45.020

Reputation: 889

1Look into deconvolution, it minimizes the L2 differnce of a convolution given a filter and activation map, and then you can approximate inverses of the activations (though understand you do lose information like at RELU in the negative region) so itll only show parts. But yes you can construct pieces of image given the features and featurizer – mshlis – 2019-08-23T16:50:44.373

Can you provide more details? Like a starter reference? – Tina J – 2019-08-23T18:50:42.223



The model (that I know of) which most resembles your description is the auto-encoder, which is trained to learn a compact representation (a vector) of the input, which can later be used to reconstruct the original input. In a certain way, this compact representation (implicitly) encodes the most important features of the input. In particular, you may be looking for denoising auto-encoders.


Posted 2019-08-22T19:50:45.020

Reputation: 19 783

Ummm auto encoders or GANs are probably the way. Can you describe what they are? I'm new. – Tina J – 2019-08-22T21:25:43.083


@TinaJ In the paper that introduced the denoising auto-encoder (that I am linking you to in my answer), Extracting and Composing Robust Features with Denoising Autoencoders, the basic auto-encoder is also described in section 2.2 The Basic Autoencoder. Have a look at that description. If you have any specific questions related to the auto-encoder, I think you should ask them in another post, and I will be glad to help you, if I am able to ;)

– nbro – 2019-08-22T21:29:00.543

No problem. Thanks. Will do. – Tina J – 2019-08-22T21:39:41.017

one thing I didn't get is why we should go for "denoising" version of autoencoders. That's for when we have corrupted inputs, right? How can it be more helpful here? – Tina J – 2019-08-23T21:07:21.557

@TinaJ Yes, right. I just thought you were looking for something like that. But maybe you're just looking for a "regular" auto-encoder. There are other auto-encoders, such as variational auto-encoder (which might be harder to understand, given that it involves more mathematics). You might want to have a look at that too. – nbro – 2019-08-23T22:35:10.680