Different number of images in classes

1

I am working on deep learning CNN project. The dataset contains more than 500 classes and the classes have different numbers of items (images). For example, some of the classses have 5 images and some of the classes have 10 and some of the classes have 20 images and some of the classes have more then 20 imags ....

Can I use this dataset to create the CNN model ? Should the number of the images in each class be same number?

Note: I will use vgg to traon the model

Thank you

N.IT

Posted 2018-06-28T11:23:14.723

Reputation: 1 445

Why are you using VGG?? That's an overkill – Aditya – 2018-06-29T13:09:23.240

Because I will not train the model from scratch – N.IT – 2018-07-01T21:32:46.400

Answers

1

Frankly, even 50 images will not be sufficient if you are going to create and use a CNN model. If you think you want more images for you model training, then go for data augmentation. It is a process of transforming an image by a small amount (be it height, width, rotation etc or any combination of these). In this way, an image and its augmented image will differ slightly. You can find relevant article here-

https://medium.com/nanonets/how-to-use-deep-learning-when-you-have-limited-data-part-2-data-augmentation-c26971dc8ced

To answer the part that should there be same number of images in each class, there should be approximately same number. This problem is a general problem while working on classification task and there are several ways to deal with it, including simulating the data (augmentation).

I would suggest that first create a separate test set, then on the remaining train set, use data augmentation and finally create the model.

EDIT

Using a pretrained convnet is also an option, as stated in a deep learning book-

A common and highly effective approach to deep learning on small image datasets is to use a pretrained network. A pretrained network is a saved network that was previously trained on a large dataset, typically on a large-scale image-classification task. If this original dataset is large enough and general enough, then the spatial hierarchy of features learned by the pretrained network can effectively act as a generic model of the visual world, and hence its features can prove useful for many different computer vision problems, even though these new problems may involve completely different classes than those of the original task. For instance, you might train a network on ImageNet (where classes are mostly animals and everyday objects) and then repurpose this trained network for something as remote as identifying furniture items in images.

Ankit Seth

Posted 2018-06-28T11:23:14.723

Reputation: 1 569

Do you mean I have to increase the number of images in the classes that have low amount of images? – N.IT – 2018-06-29T12:57:02.930

1Yeah, pretty much. If this does not work to your expectation, try creating more images for all classes (even for that which has highest number of images). – Ankit Seth – 2018-06-29T13:15:24.680

Should all classes have the same number of images ? – N.IT – 2018-06-29T14:20:57.383

1Preferably approximately same number. (like 90, 100, 94, 101 etc). These numbers are just for approximation, not the exact quantity. – Ankit Seth – 2018-06-29T15:00:50.363

1

You have a very low amount of images to create a CNN from scratch. You may be able to train your model via transfer learning from a pre-trained model, but you still may have too small of a dataset.

See something like this: https://www.analyticsvidhya.com/blog/2017/06/transfer-learning-the-art-of-fine-tuning-a-pre-trained-model/

Basically, you can take an image model that has been trained on tons of images, and either use their weights to seed or (more likely) just retrain the fully connected dense layers.

Andy M

Posted 2018-06-28T11:23:14.723

Reputation: 400

But he has to first check if his image categories are in the pre-trained model or not. Even if one category is not present, he can't use that. – Ankit Seth – 2018-06-28T11:46:01.590

1I don't see what that would need to be the case. We're essentially using the pre-trained CNN as a feature extractor, so as long as the images are somewhat similar to the original training set, it would still be useful. – Andy M – 2018-06-28T11:49:09.547

I will use vgg net to create the model – N.IT – 2018-06-29T12:54:04.000