Are there any general tips for troubleshooting a VAE when apparently it is not learning?

2

I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the output) are a 12D vector. I tried several sizes for the latent space, but, for some reason, it's not training. From the beginning, the KL loss in almost zero (around 1e-10), while the reconstruction loss (MSE for Gaussian distribution) is around 1, and they basically vary around these values without learning anything further.

Are there any general tips for troubleshooting a VAE (I never trained one before)?

I am pretty sure that the code is right and the data for sure has a background and signal (the ratio is 10:1), so I am not really sure what I am doing wrong.

Bill

Posted 2019-11-15T03:08:08.313

Reputation: 21

Hello bill welcome to AI:SE! Maybe try checking on the gradients to see if they are vanishing? If you ' adjusted the input and output to match what I need' without adjusting the quantity of layers and ensuring that the gradients don't vanish or explode anywhere in the middle then that could be the issue. – Michael Hearn – 2019-11-15T06:23:19.620

@MichaelHearn thank you for the idea! I will look into it. However, my NN is 6-7 layers deep, so I doubt there is a major gradient problem (I trained a lot deeper ones without BatchNorm or anything special, without any problem). – Bill – 2019-11-15T15:06:13.017

No answers