## What are examples of approaches to dimensionality reduction of feature vectors?

4

Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.

I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine similarity/euclidean distance matrix calculation.

I have already come up with the following two methods in my literature review:

1. Principal Component Analysis (PCA) + Whitening
2. Locality Search Hashing (LSH)

Are there more approaches to perform dimensionality reduction of feature vectors? If so, what are the pros/cons of each perhaps?

3

Dimensionality reduction could be achieved by using an Autoencoder Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding.

You could assign the encoder layer output ($$L_i$$) to a desired dimension (lower than that of the input). Once trained, $$L_i$$ could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.

1Maybe this answer could be further improved if you link to a paper or implementation that shows the application of AE to reduce the dimensionality of feature vectors. If you consider images feature vectors, then, in a way, AE are commonly applied to reduce the dimensionality of images (or feature vectors), but what if the inputs are not images? – nbro – 2020-01-26T23:28:10.927

Is there any python library perform autoencoder efficiently? – Färid Alijani – 2020-02-03T06:44:42.500

1

@FäridAlijani Not that I know of. However, designing one in Keras wouldn't be much of a task. The following Keras blog might help : https://blog.keras.io/building-autoencoders-in-keras.html

2

Some examples of dimensionality reduction techniques:

• PCA
• CCA
• ICA
• SVD
• LDA
• NMF

## Non-linear methods

• Kernel PCA
• GDA
• Autoencoders
• t-SNE
• UMAP
• MVU

### Graph-based methods ("Network embedding")

• Diffusion maps
• Graph Autoencoders
• Graph-based kernel PCA
• Isomap
• LLE
• Hessian LLE
• Laplacian Eigenmaps

Though there are many more.