Should I apply image processing techniques to the inputs of convolution networks?


After working for some time with feature-based pattern recognition, I am switching to CNN to see if I can get a higher recognition rate.

In my feature-based algorithm, I do some image processing on the picture before extracting the features, such as some convolution filters to reduce noise and segmentation into the foreground and background, and finally identifying and binarization of objects.

Should I do the same image processing before feeding data into my CNN, or is it possible to feed raw data to a CNN and expect that the CNN will adapt automatically without per-image-processing steps?

Hossam Alzomor

Posted 2020-02-25T13:58:17.630

Reputation: 25



The whole interest of using deep learning-based solutions is that you don't have to do all those pre-processings, i.e. binarization, segmentation of background. CNNs, such as YOLO or FasterRCNN, can learn how to retrieve that information by themselves.


Posted 2020-02-25T13:58:17.630

Reputation: 343

1This answer can be misleading. Input preprocessing (e.g. normalization) can actually enhance a lot the performance of a CNN. Maybe you should clarify this in this answer. – nbro – 2020-02-27T16:12:05.453

I believe that in this question, he means by pre-processing doing some feature extraction: Segmentation into the foreground and background. But of course I agree that normalizing, adding noise, rotating can improve the quality of CNN based methods. And can also constitue a data augmentation – hola – 2020-02-27T17:13:55.530


The CNN should work without trying to do special feature extraction. As pointed out some pre-processing can aid in enhancing the CNN's classification results. The Keras ImageDataGenerator provides optional parameters you can set to provide pre-processing as well as provide data augmentation. One thing I know that works for sure but can be painful is cropping the images in such a way that the subject of interest occupies a high percentage of the pixels in the resultant cropped image. The cropped image can than be resized as needed. The logic here is simple. You want your CNN to train on the subject of interest (for example a bird sitting in a tree where the bird is the subject of interest). The part of the image that is not of the bird is essentially just noise making the classifier's job harder. For example say you have a 500 X 500 initial image in which the subject of interest (the bird) only takes up 10% of the pixels (25,000 pixels). Now say as input to your CNN you reduce the image size to 100 X 100. Now the (pixels that the CNN 'learns' from is down to 1000 pixels. However lets say you crop the image so that the features of the bird are preserved but the pixels of the bird in the cropped image take up 50% of the pixels. Now if you resize the cropped image to 100 X 100 , 5000 pixels of relevance are available for the network to learn from. I have done this on several data sets. In particular images of people where the subject of interest is the face. There are many programs that are effective at cropping these images so that mostly just the face appears in the cropped result. I have trained a deep CNN in one case using uncropped images and in the other with cropped images. The results are significantly better using the cropped images.

Gerry P

Posted 2020-02-25T13:58:17.630

Reputation: 500

So cropping training data to object of interest improves the recognition rate, is that valid regardless of the size of the Object of interest in validation/test data? – Hossam Alzomor – 2020-03-03T09:42:25.543

I have not conducted experiments to determine what I would call a threshold point. That is at what point you should elect to crop images based on what percentage of pixels in the image the subject of interest occupies.Clearly at 50% you will probably not gain much by cropping the images. What I have found is cropping the images does improve accuracy on data sets I have developed. where in many cases the subject occupied less than 10% of the image. – Gerry P – 2020-03-03T17:03:40.557

Gerry P, doesn't that make it harder for the detector for different sized images though? – FourierFlux – 2020-03-04T04:20:56.353

No after cropping all images you use as input to the classifier must be the same size so just resize the cropped image. Keras flow_from_directory will do that for you just specify the height and width. – Gerry P – 2020-03-04T17:42:02.077