Can I say that a trained neural network model with less parameters requires less resources during real world inference?



Let us imagine that we have two trained neural network models with different architectures (e.g., type of layers). The first model (a) uses 1D convolutional layers with fully-connected layers and has 10 million learnable prameters. The second model (b) does use 2d conv layer with and has only 1 million paramerts in total. Both model achieve equal scores on the same input data set.

Can I say that model b with less parameter is more favourable because it has less (trainable) prameters and for this reason it requires less resources (GPU-Power, Memory ...) if it would used in a real world scenario? Or does this only show that I is able to learn more effectively in constrast to model a? Are the required resource during real world inference also influenced by the layer types? How can I measure the resource utilization during inference

Just for information: I use tensorflow / keras for both models.

Thanks for your opinon.


Posted 2021-01-16T16:29:11.380

Reputation: 430

3well memory and CPU are certainly a function of num of paremeters – Nikos M. – 2021-01-16T17:31:31.887

No answers